-- Logs begin at Mon 2022-07-25 04:47:26 UTC, end at Mon 2022-07-25 05:55:57 UTC. -- Jul 25 05:22:42.405298 dv-cinder-850830-1-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Jul 25 05:22:42.414085 dv-cinder-850830-1-iscsi devstack@keystone.service[80238]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Jul 25 05:22:42.416765 dv-cinder-850830-1-iscsi devstack@keystone.service[80238]: *** Starting uWSGI 2.0.18-debian (64bit) on [Mon Jul 25 05:22:42 2022] *** Jul 25 05:22:42.416961 dv-cinder-850830-1-iscsi devstack@keystone.service[80238]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Jul 25 05:22:42.417107 dv-cinder-850830-1-iscsi devstack@keystone.service[80238]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Jul 25 05:22:42.417241 dv-cinder-850830-1-iscsi devstack@keystone.service[80238]: nodename: dv-cinder-850830-1-iscsi Jul 25 05:22:42.417241 dv-cinder-850830-1-iscsi devstack@keystone.service[80238]: machine: x86_64 Jul 25 05:22:42.417241 dv-cinder-850830-1-iscsi devstack@keystone.service[80238]: clock source: unix Jul 25 05:22:42.417241 dv-cinder-850830-1-iscsi devstack@keystone.service[80238]: pcre jit disabled Jul 25 05:22:42.417241 dv-cinder-850830-1-iscsi devstack@keystone.service[80238]: detected number of CPU cores: 4 Jul 25 05:22:42.417241 dv-cinder-850830-1-iscsi devstack@keystone.service[80238]: current working directory: / Jul 25 05:22:42.417241 dv-cinder-850830-1-iscsi devstack@keystone.service[80238]: detected binary path: /usr/bin/uwsgi-core Jul 25 05:22:42.417241 dv-cinder-850830-1-iscsi devstack@keystone.service[80238]: your processes number limit is 47680 Jul 25 05:22:42.417241 dv-cinder-850830-1-iscsi devstack@keystone.service[80238]: your memory page size is 4096 bytes Jul 25 05:22:42.417241 dv-cinder-850830-1-iscsi devstack@keystone.service[80238]: detected max file descriptor number: 2048 Jul 25 05:22:42.417241 dv-cinder-850830-1-iscsi devstack@keystone.service[80238]: lock engine: pthread robust mutexes Jul 25 05:22:42.417839 dv-cinder-850830-1-iscsi devstack@keystone.service[80238]: thunder lock: enabled Jul 25 05:22:42.417839 dv-cinder-850830-1-iscsi devstack@keystone.service[80238]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Jul 25 05:22:42.417839 dv-cinder-850830-1-iscsi devstack@keystone.service[80238]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Jul 25 05:22:42.461352 dv-cinder-850830-1-iscsi devstack@keystone.service[80238]: Python main interpreter initialized at 0x56288cfb09c0 Jul 25 05:22:42.461352 dv-cinder-850830-1-iscsi devstack@keystone.service[80238]: python threads support enabled Jul 25 05:22:42.461352 dv-cinder-850830-1-iscsi devstack@keystone.service[80238]: your server socket listen backlog is limited to 100 connections Jul 25 05:22:42.461352 dv-cinder-850830-1-iscsi devstack@keystone.service[80238]: your mercy for graceful operations on workers is 90 seconds Jul 25 05:22:42.461681 dv-cinder-850830-1-iscsi devstack@keystone.service[80238]: mapped 403077 bytes (393 KB) for 2 cores Jul 25 05:22:42.461757 dv-cinder-850830-1-iscsi devstack@keystone.service[80238]: *** Operational MODE: preforking *** Jul 25 05:22:42.462034 dv-cinder-850830-1-iscsi devstack@keystone.service[80238]: *** uWSGI is running in multiple interpreter mode *** Jul 25 05:22:42.462034 dv-cinder-850830-1-iscsi devstack@keystone.service[80238]: spawned uWSGI master process (pid: 80238) Jul 25 05:22:42.462232 dv-cinder-850830-1-iscsi systemd[1]: Started Devstack devstack@keystone.service. Jul 25 05:22:42.468814 dv-cinder-850830-1-iscsi devstack@keystone.service[80238]: spawned uWSGI worker 1 (pid: 80248, cores: 1) Jul 25 05:22:42.468814 dv-cinder-850830-1-iscsi devstack@keystone.service[80238]: spawned uWSGI worker 2 (pid: 80249, cores: 1) Jul 25 05:22:42.468814 dv-cinder-850830-1-iscsi devstack@keystone.service[80238]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Jul 25 05:22:47.974691 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 25 05:22:47.974691 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: from cryptography.utils import int_from_bytes Jul 25 05:22:47.975486 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 25 05:22:47.975486 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: from cryptography.utils import int_from_bytes Jul 25 05:22:48.024527 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Jul 25 05:22:48.024734 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Jul 25 05:22:48.024955 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] command line args: [] {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Jul 25 05:22:48.025241 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Jul 25 05:22:48.025450 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] ================================================================================ {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Jul 25 05:22:48.025718 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] admin_token = **** {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 05:22:48.026023 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] config_dir = [] {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 05:22:48.026237 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 05:22:48.026458 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] config_source = [] {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 05:22:48.026680 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] debug = True {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 05:22:48.026911 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 05:22:48.028072 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 05:22:48.028072 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 05:22:48.028072 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 05:22:48.028072 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 05:22:48.028320 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] list_limit = None {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 05:22:48.028520 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] log_config_append = None {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 05:22:48.028737 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 05:22:48.028976 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] log_dir = None {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 05:22:48.029196 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] log_file = None {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 05:22:48.029470 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 05:22:48.029760 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 05:22:48.030041 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 05:22:48.030324 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 05:22:48.030528 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 05:22:48.030725 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 05:22:48.030996 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 05:22:48.031263 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 05:22:48.031548 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 05:22:48.031816 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 05:22:48.032085 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 05:22:48.032352 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 05:22:48.032624 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 05:22:48.032933 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 05:22:48.033237 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 05:22:48.033780 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.33/identity {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 05:22:48.034000 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] publish_errors = False {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 05:22:48.034271 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 05:22:48.034537 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 05:22:48.034757 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 05:22:48.035557 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 05:22:48.035866 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 05:22:48.036143 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] standard_threads = False {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 05:22:48.036441 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 05:22:48.036721 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 05:22:48.036971 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 05:22:48.037194 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] use_journal = False {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 05:22:48.037410 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] use_json = False {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 05:22:48.037621 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] use_stderr = False {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 05:22:48.037836 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] use_syslog = False {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 05:22:48.039384 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 05:22:48.039817 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.040229 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.040657 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.041081 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.041537 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.041933 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.042387 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.042781 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.043235 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.043639 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.044090 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.044479 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.044821 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.045073 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.045329 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.045585 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.045829 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.046246 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.046613 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.046868 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.047113 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.047375 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.047643 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.047902 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.048158 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.048400 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.048635 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.048896 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.049138 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.049393 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.049644 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.050029 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.050422 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.050709 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.051488 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.051738 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.051994 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.052236 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.052488 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.052731 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.052938 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 25 05:22:48.052938 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: from cryptography.utils import int_from_bytes Jul 25 05:22:48.053491 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.053491 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.053621 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 25 05:22:48.053621 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: from cryptography.utils import int_from_bytes Jul 25 05:22:48.053736 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.053736 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.053896 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.054148 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.054385 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.054625 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.054899 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.055098 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.055337 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.055589 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.055823 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.056064 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.056304 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.056544 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.056804 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.057046 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.057280 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.057533 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.057774 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.058015 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.058254 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.058517 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.058757 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.058991 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.059232 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.059471 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.059746 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.059988 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.060239 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.060490 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.060729 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.061004 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.061245 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.061495 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.061736 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.061978 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.062225 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.062465 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.062704 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.062941 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.063181 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.063420 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.063684 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.063928 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.064174 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.064416 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.064667 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.064934 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.065174 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.065412 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.065646 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.065879 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.066112 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.066390 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.066629 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.066874 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.067298 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.067557 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.067802 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.068322 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.068590 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.068862 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.069102 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] auth.external = None {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.069338 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.069642 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.069880 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.070115 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] auth.password = None {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.070348 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] auth.token = None {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.070592 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.070831 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.071080 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.071315 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.071566 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.071816 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.072052 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.072290 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] credential.caching = True {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.072540 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.072813 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.073070 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.073311 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.073558 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.073796 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.074061 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.074318 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.074559 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.074815 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.075106 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.075356 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.075626 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.075872 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.076131 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.076370 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] federation.caching = True {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.076609 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.076886 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.077138 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.077370 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.077620 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.077861 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.078126 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.078374 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.078635 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.078877 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.079129 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.079370 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] identity.caching = True {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.079623 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.079873 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.080112 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.080350 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.080719 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.080991 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.081238 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.081496 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.081767 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.082045 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.082425 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.082667 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.082923 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.083181 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.083434 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.083699 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.083953 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.084231 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.084475 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.084719 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.084986 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.085228 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.085467 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.085709 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.085956 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.086200 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.086457 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.086691 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.086943 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.087197 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.087437 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.087689 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.087941 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.088178 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.088426 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.088661 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.088929 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.089172 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.089419 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.089659 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.089901 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.090155 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.090409 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.090645 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.090879 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.091131 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.091383 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.091623 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.091863 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.092100 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.092335 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] ldap.user = None {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.092579 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.092868 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.093106 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.093357 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.093608 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.093858 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.094097 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.094331 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.094570 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.094808 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.095049 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.095289 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.095542 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.095792 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.096041 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.096297 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.096548 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.096807 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.097058 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.097302 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.098048 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.098298 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.098592 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.098796 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.099050 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.099294 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.099554 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.099792 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.100040 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.100281 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.100522 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.100763 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.101044 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.101179 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Jul 25 05:22:48.101291 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.101394 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Jul 25 05:22:48.101557 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.101635 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] command line args: [] {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Jul 25 05:22:48.101755 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.101837 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Jul 25 05:22:48.101998 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] resource.caching = True {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.102080 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] ================================================================================ {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Jul 25 05:22:48.102253 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.102334 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] admin_token = **** {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 05:22:48.102505 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.102585 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] config_dir = [] {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 05:22:48.102727 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 05:22:48.102809 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.102923 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] config_source = [] {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 05:22:48.103014 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.103100 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] debug = True {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 05:22:48.103225 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.103338 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 05:22:48.103467 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.103585 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 05:22:48.103717 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.103797 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 05:22:48.103960 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.104041 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 05:22:48.104202 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.104289 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 05:22:48.104440 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] role.caching = True {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.104521 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] list_limit = None {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 05:22:48.104691 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] role.driver = sql {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.104803 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] log_config_append = None {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 05:22:48.104945 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 05:22:48.105026 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.105136 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] log_dir = None {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 05:22:48.105240 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.105316 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] log_file = None {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 05:22:48.105435 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.105559 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 05:22:48.105686 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.105818 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 05:22:48.105939 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.106083 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 05:22:48.106190 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.106331 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 05:22:48.106442 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.106691 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.106771 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 05:22:48.106915 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 05:22:48.107009 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.107165 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 05:22:48.107247 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.107412 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 05:22:48.107494 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.107646 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 05:22:48.107735 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.107887 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 05:22:48.107968 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.108120 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 05:22:48.108201 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.108352 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 05:22:48.108543 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.108626 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 05:22:48.108802 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.108888 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 05:22:48.109066 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.109148 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 05:22:48.109324 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.109579 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.109660 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.33/identity {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 05:22:48.109758 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] publish_errors = False {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 05:22:48.109847 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.109989 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 05:22:48.110071 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.110219 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 05:22:48.110301 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.110382 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 05:22:48.110559 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.110650 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 05:22:48.110805 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.110886 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 05:22:48.111042 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.111124 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] standard_threads = False {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 05:22:48.111278 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.111361 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 05:22:48.111514 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.111594 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 05:22:48.111689 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 05:22:48.111769 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.111852 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] use_journal = False {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 05:22:48.112018 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.112099 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] use_json = False {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 05:22:48.112193 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] use_stderr = False {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 05:22:48.112273 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.112357 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] use_syslog = False {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 05:22:48.112497 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.112578 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 05:22:48.112737 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.112837 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.113009 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.113101 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.113878 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] token.caching = True {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.114011 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.114155 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.114414 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.114526 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.114658 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.114774 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.114906 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.115029 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.115160 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.115282 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.115402 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.115522 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.115651 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.115774 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.115898 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.116024 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.116188 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.116266 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.116378 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.116499 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.116621 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.116748 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.116872 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.117014 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.117126 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.117262 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.117379 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.117511 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.117601 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.117749 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.117850 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.118003 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.118155 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.118259 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.118433 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] database.connection = **** {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.118527 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.118684 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.118767 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.118936 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.119017 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.119178 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.119268 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.119421 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.119512 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.119659 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.119740 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.119900 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.119993 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.120136 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.120216 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.120366 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.120447 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.120604 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.120693 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.120876 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.120961 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.121125 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.121204 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.121354 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.121451 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.121588 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.121669 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.121827 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.121907 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.122074 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.122161 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.122304 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.122385 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.122547 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.122634 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.122781 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.122869 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.122946 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Jul 25 05:22:48.123389 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.123634 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.123868 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.124101 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.124336 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.124567 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.124826 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.125081 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.125321 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.125560 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.125793 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.126029 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.126263 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.126512 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.126744 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.126983 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.127221 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.127457 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.127695 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.127934 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.128165 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.128414 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.128650 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.129066 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.129508 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.129804 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.130048 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.130285 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.130532 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.130773 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.131047 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.131288 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.131539 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.131786 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.132024 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.132261 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.132500 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.132751 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.133016 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.133256 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.133502 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.133741 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.133977 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.134214 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.134452 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.134689 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.134951 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.135193 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.135439 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.135676 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.135920 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.136156 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.136393 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.136629 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.136884 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.137118 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.137350 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.137587 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.137825 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.138067 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.138303 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.138551 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.138787 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.139046 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.139307 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.139549 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.139783 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] auth.external = None {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.140015 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.140314 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.140548 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.140805 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] auth.password = None {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.141030 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] auth.token = None {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.141270 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.141506 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.141761 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.141994 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.142242 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.142488 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.142721 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.142956 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] credential.caching = True {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.143201 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.143451 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.143701 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.143939 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.144184 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.144988 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.145474 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.145791 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.146035 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.146294 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.146580 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.146830 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.147093 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.147338 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.147595 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.147833 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] federation.caching = True {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.148070 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.148318 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.148566 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.148816 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.149071 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.149309 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.149562 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.149800 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.150056 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.150295 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.150543 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.150779 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] identity.caching = True {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.151028 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.151276 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.151511 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.151750 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.151997 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.152228 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.152466 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.152851 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.153115 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.153354 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.153589 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.153822 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.154068 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.154202 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80249) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 05:22:48.154319 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.154438 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80249) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 05:22:48.154569 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.154652 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80249) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 05:22:48.154827 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.155080 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.155355 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.155599 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.155846 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.156081 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.156323 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.156557 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.156813 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.157062 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.157305 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.157557 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.157792 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.158050 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.158301 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.158540 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.158787 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.159036 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.159268 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.159506 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.159738 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.159976 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.161122 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.161480 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.161727 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.161970 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.162224 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.162477 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.162714 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.162956 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.163210 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.163460 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.163556 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80249) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 05:22:48.163699 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.163937 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.164173 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.164405 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] ldap.user = None {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.164647 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.164945 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.165181 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.165431 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.165681 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.165928 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.166164 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.166395 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.166636 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.166871 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.167108 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.167337 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.167583 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.167830 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.168075 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.168322 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.168567 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.168816 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.169067 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.169307 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.169544 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.169781 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.170016 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.170266 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.170515 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.170753 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.171007 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.171238 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.171482 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.171720 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.171954 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.172191 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.172436 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.172684 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.172942 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.173181 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.173420 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] resource.caching = True {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.173670 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.173918 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.174155 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.174404 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.174655 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.174890 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.175137 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.175375 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.175615 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.176066 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] role.caching = True {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.176531 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] role.driver = sql {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.176981 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.177464 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.177827 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.178090 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.178349 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.178607 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.178865 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.179120 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.179375 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.179614 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.179866 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.180117 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.180368 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.180619 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.181071 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.181322 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.181579 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.181832 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.182083 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.182334 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.182572 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.182836 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.183094 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.183337 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.183571 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.183806 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.184039 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.184292 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.184550 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.184831 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.185074 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.185311 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.185544 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.185782 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] token.caching = True {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.186018 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.186267 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.186508 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.186768 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.187017 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.187256 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.187500 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.187743 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.187992 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.188230 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.188470 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.188704 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.188976 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.189224 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.189455 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.189697 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.190004 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.190271 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] database.connection = **** {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.190518 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.190767 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.191006 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.191125 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80249) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 05:22:48.191230 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.191407 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80249) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 05:22:48.191622 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.192010 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.192256 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.192498 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.192747 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.193267 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.193514 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.193751 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.194001 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.194238 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.194483 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.194715 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.194955 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.195191 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:22:48.195364 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Jul 25 05:22:48.221682 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80248) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 05:22:48.221914 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80248) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 05:22:48.222143 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80248) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 05:22:48.231300 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80248) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 05:22:48.250073 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80248) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 05:22:48.250316 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80248) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 05:22:48.332627 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80249) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 05:22:48.379008 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80248) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 05:22:48.383612 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80249) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 05:22:48.383852 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80249) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 05:22:48.415228 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80248) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 05:22:48.415473 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80248) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 05:22:48.448736 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80249) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 05:22:48.449001 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80249) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 05:22:48.449227 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80249) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 05:22:48.449447 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80249) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 05:22:48.449666 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80249) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 05:22:48.449883 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80249) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 05:22:48.458356 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80248) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 05:22:48.458591 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80248) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 05:22:48.458819 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80248) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 05:22:48.459041 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80248) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 05:22:48.459261 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80248) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 05:22:48.459476 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80248) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 05:22:48.474887 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80249) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 05:22:48.475124 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80249) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 05:22:48.475355 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80249) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 05:22:48.475584 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80249) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 05:22:48.475804 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80249) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 05:22:48.476029 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80249) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 05:22:48.476236 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80249) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 05:22:48.476938 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80248) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 05:22:48.477177 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80248) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 05:22:48.477404 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80248) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 05:22:48.477624 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80248) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 05:22:48.477843 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80248) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 05:22:48.478058 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80248) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 05:22:48.478273 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80248) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 05:22:48.487356 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80249) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 05:22:48.488986 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80248) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 05:22:48.499979 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80249) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 05:22:48.500214 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80249) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 05:22:48.500442 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80249) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 05:22:48.500571 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80248) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 05:22:48.500716 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80249) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 05:22:48.500875 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80248) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 05:22:48.501003 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80249) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 05:22:48.501140 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80248) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 05:22:48.501268 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80249) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 05:22:48.501397 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80248) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 05:22:48.501521 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80249) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 05:22:48.501645 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80248) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 05:22:48.501645 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80248) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 05:22:48.501890 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80248) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 05:22:48.519038 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80249) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 05:22:48.519250 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80249) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 25 05:22:48.519689 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80249) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 05:22:48.519893 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80249) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 25 05:22:48.520033 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80248) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 05:22:48.520232 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80249) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 05:22:48.520360 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80248) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 25 05:22:48.520445 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80249) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 05:22:48.520587 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80248) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 05:22:48.520671 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80249) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 05:22:48.520818 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80248) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 25 05:22:48.520950 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80249) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 05:22:48.521097 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80248) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 05:22:48.521361 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80248) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 05:22:48.521622 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80248) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 05:22:48.521877 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80248) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 05:22:48.531982 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80249) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 05:22:48.532263 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80249) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 05:22:48.532529 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80249) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 05:22:48.532834 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80249) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 05:22:48.532998 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80248) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 05:22:48.533275 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80248) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 05:22:48.533543 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80248) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 05:22:48.533823 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80248) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 05:22:48.541511 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80249) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 25 05:22:48.542789 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80248) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 25 05:22:48.544823 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80249) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 25 05:22:48.545109 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80249) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 05:22:48.545395 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80249) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 05:22:48.545790 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80249) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 05:22:48.546079 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80249) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 05:22:48.546174 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80248) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 25 05:22:48.546348 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80249) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 05:22:48.546431 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80248) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 05:22:48.546623 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80249) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 05:22:48.546704 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80248) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 05:22:48.546900 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80249) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 05:22:48.546983 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80248) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 05:22:48.547173 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80249) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 05:22:48.547254 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80248) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 05:22:48.547431 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80248) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 05:22:48.547511 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80249) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 05:22:48.547708 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80249) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 05:22:48.547810 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80248) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 05:22:48.548088 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80248) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 05:22:48.548362 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80248) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 05:22:48.548639 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80248) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 05:22:48.548941 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80248) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 05:22:48.575171 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80249) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 05:22:48.576813 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80249) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 25 05:22:48.576901 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80248) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 05:22:48.577099 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80249) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 05:22:48.578553 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80248) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 25 05:22:48.578835 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80248) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 05:22:48.582534 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80249) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 25 05:22:48.582857 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80249) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 05:22:48.583104 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80249) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 05:22:48.584507 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80248) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 25 05:22:48.584810 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80248) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 05:22:48.585089 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80248) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 05:22:48.590418 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80249) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 25 05:22:48.590702 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80249) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 05:22:48.592316 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80248) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 25 05:22:48.592600 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80248) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 05:22:48.595333 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80249) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 25 05:22:48.595625 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80249) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 05:22:48.595911 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80249) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 05:22:48.596198 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80249) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 05:22:48.596478 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80249) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 05:22:48.596753 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80249) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 05:22:48.597053 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80249) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 05:22:48.597134 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80248) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 25 05:22:48.597325 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80249) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 05:22:48.598041 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80248) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 05:22:48.598496 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80248) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 05:22:48.598918 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80248) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 05:22:48.599200 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80248) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 05:22:48.599476 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80248) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 05:22:48.599751 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80248) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 05:22:48.600024 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80248) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 05:22:48.617107 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80249) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 05:22:48.617387 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80249) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 05:22:48.619415 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80248) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 05:22:48.619694 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80248) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 05:22:48.622002 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80249) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 25 05:22:48.624334 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80248) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 25 05:22:48.625665 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80249) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 05:22:48.625965 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80249) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 05:22:48.627958 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80248) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 05:22:48.628253 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80248) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 05:22:48.631098 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80249) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 25 05:22:48.633714 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80248) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 25 05:22:48.634759 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80249) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 25 05:22:48.637350 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80248) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 25 05:22:48.638590 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80249) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 05:22:48.638894 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80249) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 05:22:48.639189 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80249) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 05:22:48.639478 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80249) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 05:22:48.639766 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80249) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 05:22:48.640050 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80249) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 05:22:48.641120 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80248) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 05:22:48.641421 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80248) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 05:22:48.641711 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80248) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 05:22:48.641991 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80248) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 05:22:48.642280 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80248) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 05:22:48.642557 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80248) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 05:22:48.659121 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80249) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 25 05:22:48.659495 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80249) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 05:22:48.659769 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80249) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 05:22:48.660649 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80249) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 05:22:48.662757 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80248) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 25 05:22:48.663040 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80248) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 05:22:48.663309 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80248) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 05:22:48.663578 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80248) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 05:22:48.668687 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80249) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 05:22:48.670451 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80249) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 05:22:48.670730 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80249) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 05:22:48.671581 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80248) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 05:22:48.673313 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80248) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 05:22:48.673590 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80248) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 05:22:48.673889 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80249) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 25 05:22:48.674172 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80249) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 05:22:48.674457 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80249) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 05:22:48.674855 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80249) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 05:22:48.675147 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80249) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 05:22:48.677642 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80248) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 25 05:22:48.677923 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80248) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 05:22:48.678203 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80248) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 05:22:48.678486 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80248) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 05:22:48.678766 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80248) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 05:22:48.689351 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80249) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 25 05:22:48.689633 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80249) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 05:22:48.689914 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80249) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 05:22:48.690202 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80249) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 05:22:48.690482 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80249) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 05:22:48.690773 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80249) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 05:22:48.691051 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80249) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 05:22:48.692710 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80248) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 25 05:22:48.693269 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80248) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 05:22:48.693559 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80248) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 05:22:48.693850 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80248) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 05:22:48.694132 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80248) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 05:22:48.694417 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80248) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 05:22:48.694694 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80248) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 05:22:48.710792 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80249) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 25 05:22:48.714085 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80249) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 25 05:22:48.714231 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80248) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 25 05:22:48.717377 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80248) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 25 05:22:48.717512 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80249) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 05:22:48.719035 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80249) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 05:22:48.720657 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80248) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 05:22:48.720826 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80249) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 25 05:22:48.720957 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80249) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 05:22:48.721243 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80249) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 05:22:48.722293 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80248) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 05:22:48.724962 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80248) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 25 05:22:48.725243 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80248) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 05:22:48.725533 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80248) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 05:22:48.729324 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80249) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 05:22:48.730955 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80249) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 25 05:22:48.732938 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80248) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 05:22:48.734213 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80249) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 05:22:48.734505 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80249) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 05:22:48.734635 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80248) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 25 05:22:48.734789 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80249) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 05:22:48.735080 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80249) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 05:22:48.737831 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80248) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 05:22:48.738209 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80248) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 05:22:48.738682 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80248) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 05:22:48.739046 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80248) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 05:22:48.744473 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80249) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 25 05:22:48.744763 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80249) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 05:22:48.745072 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80249) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 05:22:48.748150 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80248) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 25 05:22:48.748433 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80248) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 05:22:48.748841 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80248) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 05:22:48.752944 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80249) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 25 05:22:48.753235 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80249) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 05:22:48.753517 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80249) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 05:22:48.754323 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80249) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 05:22:48.754598 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80249) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 05:22:48.755062 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80249) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 05:22:48.755346 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80249) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 05:22:48.755632 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80249) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 05:22:48.755906 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80249) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 05:22:48.756179 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80249) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 05:22:48.756458 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80249) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 05:22:48.756734 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80249) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 05:22:48.757031 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80249) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 05:22:48.757301 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80249) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 05:22:48.757591 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80248) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 25 05:22:48.757869 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80248) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 05:22:48.758147 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80248) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 05:22:48.758420 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80248) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 05:22:48.758688 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80248) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 05:22:48.758964 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80248) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 05:22:48.759238 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80248) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 05:22:48.759524 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80248) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 05:22:48.759801 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80248) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 05:22:48.760068 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80248) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 05:22:48.760336 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80248) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 05:22:48.760596 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80248) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 05:22:48.760879 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80248) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 05:22:48.761148 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80248) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 05:22:48.795796 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x56288cfb09c0 pid: 80249 (default app) Jul 25 05:22:48.799202 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df15454e-6da3-4aa0-af72-ff67a9b185e2 None None] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:22:48.799503 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df15454e-6da3-4aa0-af72-ff67a9b185e2 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:22:48.799788 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df15454e-6da3-4aa0-af72-ff67a9b185e2 None None] PATH_INFO: `/v3` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:22:48.799949 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x56288cfb09c0 pid: 80248 (default app) Jul 25 05:22:48.804108 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1/1] 10.222.0.33 () {50 vars in 791 bytes} [Mon Jul 25 05:22:48 2022] GET /identity/v3/ => generated 255 bytes in 8 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:22:56.470110 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7056c56c-6dac-45ee-b91d-93d34a1cd9c3 None None] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:22:56.470689 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7056c56c-6dac-45ee-b91d-93d34a1cd9c3 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:22:56.470689 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7056c56c-6dac-45ee-b91d-93d34a1cd9c3 None None] PATH_INFO: `/` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:22:56.475760 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1/2] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jul 25 05:22:56 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 25 05:22:56.480951 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef31283-0f0a-4f25-897c-cd327448d2c3 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:22:56.481388 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef31283-0f0a-4f25-897c-cd327448d2c3 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:22:56.481508 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef31283-0f0a-4f25-897c-cd327448d2c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:22:56.552680 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: INFO dbcounter [None req-1ef31283-0f0a-4f25-897c-cd327448d2c3 None None] Registered counter for database keystone Jul 25 05:22:56.574464 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG oslo_db.sqlalchemy.engines [None req-1ef31283-0f0a-4f25-897c-cd327448d2c3 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80249) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jul 25 05:22:56.580643 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writer thread running {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jul 25 05:22:56.760656 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG passlib.handlers.bcrypt [None req-1ef31283-0f0a-4f25-897c-cd327448d2c3 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80249) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Jul 25 05:22:56.761024 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG passlib.handlers.bcrypt [None req-1ef31283-0f0a-4f25-897c-cd327448d2c3 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80249) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Jul 25 05:22:56.815717 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-1ef31283-0f0a-4f25-897c-cd327448d2c3 None None] MFA Rules not processed for user `1eae540242d94c10b0ad33960a855f08`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:22:56.817610 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-1ef31283-0f0a-4f25-897c-cd327448d2c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:22:56.937136 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 2/3] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jul 25 05:22:56 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 457 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Jul 25 05:22:57.110802 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7056c56c-6dac-45ee-b91d-93d34a1cd9c3 None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:22:57.180622 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: INFO dbcounter [None req-7056c56c-6dac-45ee-b91d-93d34a1cd9c3 None None] Registered counter for database keystone Jul 25 05:22:57.199731 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG oslo_db.sqlalchemy.engines [None req-7056c56c-6dac-45ee-b91d-93d34a1cd9c3 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80248) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jul 25 05:22:57.206454 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writer thread running {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jul 25 05:22:57.323133 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7056c56c-6dac-45ee-b91d-93d34a1cd9c3 None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:22:57.327389 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5924bcc9-20c5-4ad3-a3d5-920297bea8e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:22:57.328042 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5924bcc9-20c5-4ad3-a3d5-920297bea8e9 admin admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:22:57.328329 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5924bcc9-20c5-4ad3-a3d5-920297bea8e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:22:57.328607 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5924bcc9-20c5-4ad3-a3d5-920297bea8e9 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:22:57.337348 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5924bcc9-20c5-4ad3-a3d5-920297bea8e9 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:22:57.388712 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: WARNING keystone.common.rbac_enforcer.enforcer [None req-5924bcc9-20c5-4ad3-a3d5-920297bea8e9 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Jul 25 05:22:57.389028 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5924bcc9-20c5-4ad3-a3d5-920297bea8e9 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:22:57.398985 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 2/4] 10.222.0.33 () {58 vars in 1324 bytes} [Mon Jul 25 05:22:57 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 289 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 25 05:22:57.405059 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef31283-0f0a-4f25-897c-cd327448d2c3 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:22:57.420380 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef31283-0f0a-4f25-897c-cd327448d2c3 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:22:57.425280 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-302ac279-e72b-4607-9257-881259c1f598 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:22:57.426443 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302ac279-e72b-4607-9257-881259c1f598 admin admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:22:57.426735 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302ac279-e72b-4607-9257-881259c1f598 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:22:57.427014 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302ac279-e72b-4607-9257-881259c1f598 admin admin] PATH_INFO: `/v3/projects/85f4644d3bc94e89a0061a116652482e` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:22:57.428509 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-302ac279-e72b-4607-9257-881259c1f598 admin admin] RBAC: Authorizing `identity:get_project(project_id=85f4644d3bc94e89a0061a116652482e)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:22:57.483217 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: WARNING keystone.common.rbac_enforcer.enforcer [None req-302ac279-e72b-4607-9257-881259c1f598 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Jul 25 05:22:57.483777 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-302ac279-e72b-4607-9257-881259c1f598 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:22:57.489398 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 3/5] 10.222.0.33 () {56 vars in 1207 bytes} [Mon Jul 25 05:22:57 2022] GET /identity/v3/projects/85f4644d3bc94e89a0061a116652482e => generated 344 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:22:59.485638 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a8198e-210c-4be2-ba60-3a489dbd37a6 None None] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:22:59.486186 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a8198e-210c-4be2-ba60-3a489dbd37a6 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:22:59.486186 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a8198e-210c-4be2-ba60-3a489dbd37a6 None None] PATH_INFO: `/` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:22:59.490786 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 3/6] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jul 25 05:22:59 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 25 05:22:59.496327 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8b4cf0-b995-420e-b6d2-2d2689012699 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:22:59.496720 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8b4cf0-b995-420e-b6d2-2d2689012699 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:22:59.496937 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8b4cf0-b995-420e-b6d2-2d2689012699 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:22:59.516048 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-5f8b4cf0-b995-420e-b6d2-2d2689012699 None None] MFA Rules not processed for user `1eae540242d94c10b0ad33960a855f08`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:22:59.517606 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-5f8b4cf0-b995-420e-b6d2-2d2689012699 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:22:59.526122 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 4/7] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jul 25 05:22:59 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Jul 25 05:22:59.548236 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a8198e-210c-4be2-ba60-3a489dbd37a6 None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:22:59.560757 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a8198e-210c-4be2-ba60-3a489dbd37a6 None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:22:59.564971 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f80600-4165-4cfb-ba45-2d0c18a0adf4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:22:59.565601 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f80600-4165-4cfb-ba45-2d0c18a0adf4 admin admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:22:59.565885 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f80600-4165-4cfb-ba45-2d0c18a0adf4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:22:59.566160 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f80600-4165-4cfb-ba45-2d0c18a0adf4 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:22:59.574093 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03f80600-4165-4cfb-ba45-2d0c18a0adf4 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:22:59.575850 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03f80600-4165-4cfb-ba45-2d0c18a0adf4 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:22:59.585172 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 4/8] 10.222.0.33 () {58 vars in 1324 bytes} [Mon Jul 25 05:22:59 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 25 05:22:59.590476 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f8b4cf0-b995-420e-b6d2-2d2689012699 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:22:59.600453 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f8b4cf0-b995-420e-b6d2-2d2689012699 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:22:59.604802 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdea691d-3967-4acb-a5c4-7b3b020dd117 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:22:59.605700 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdea691d-3967-4acb-a5c4-7b3b020dd117 admin admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:22:59.606009 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdea691d-3967-4acb-a5c4-7b3b020dd117 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:22:59.606285 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdea691d-3967-4acb-a5c4-7b3b020dd117 admin admin] PATH_INFO: `/v3/users/1eae540242d94c10b0ad33960a855f08` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:22:59.607671 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdea691d-3967-4acb-a5c4-7b3b020dd117 admin admin] RBAC: Authorizing `identity:get_user(user_id=1eae540242d94c10b0ad33960a855f08)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:22:59.609518 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdea691d-3967-4acb-a5c4-7b3b020dd117 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:22:59.611091 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 5/9] 10.222.0.33 () {56 vars in 1198 bytes} [Mon Jul 25 05:22:59 2022] GET /identity/v3/users/1eae540242d94c10b0ad33960a855f08 => generated 247 bytes in 21 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:23:01.670205 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354b187e-2dd1-4825-8b80-b13a07d11437 None None] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:23:01.670734 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354b187e-2dd1-4825-8b80-b13a07d11437 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:23:01.670734 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354b187e-2dd1-4825-8b80-b13a07d11437 None None] PATH_INFO: `/` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:23:01.672112 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 5/10] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jul 25 05:23:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 25 05:23:01.677285 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2889bccb-66ae-450a-81f6-6140aef6d759 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:23:01.677722 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2889bccb-66ae-450a-81f6-6140aef6d759 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:23:01.677856 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2889bccb-66ae-450a-81f6-6140aef6d759 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:23:01.694482 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-2889bccb-66ae-450a-81f6-6140aef6d759 None None] MFA Rules not processed for user `1eae540242d94c10b0ad33960a855f08`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:23:01.695674 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-2889bccb-66ae-450a-81f6-6140aef6d759 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:23:01.763961 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6a0e3d-4441-46d7-93c4-4950f52d9e51 None None] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:23:01.764246 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6a0e3d-4441-46d7-93c4-4950f52d9e51 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:23:01.765115 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6a0e3d-4441-46d7-93c4-4950f52d9e51 None None] PATH_INFO: `/` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:23:01.766143 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 6/11] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jul 25 05:23:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 25 05:23:01.771880 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5edb81a7-d4f7-4de3-8e41-6e1679608cec None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:23:01.772169 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5edb81a7-d4f7-4de3-8e41-6e1679608cec None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:23:01.772449 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5edb81a7-d4f7-4de3-8e41-6e1679608cec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:23:01.776414 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 6/12] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jul 25 05:23:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 05:23:01.800025 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2889bccb-66ae-450a-81f6-6140aef6d759 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:23:01.814892 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG passlib.handlers.bcrypt [None req-5edb81a7-d4f7-4de3-8e41-6e1679608cec None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80248) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Jul 25 05:23:01.815305 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG passlib.handlers.bcrypt [None req-5edb81a7-d4f7-4de3-8e41-6e1679608cec None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80248) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Jul 25 05:23:01.825279 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:23:01.861533 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-5edb81a7-d4f7-4de3-8e41-6e1679608cec None None] MFA Rules not processed for user `1eae540242d94c10b0ad33960a855f08`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:23:01.863117 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-5edb81a7-d4f7-4de3-8e41-6e1679608cec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:23:01.907328 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2889bccb-66ae-450a-81f6-6140aef6d759 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:23:01.940691 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-018d9921-58d6-4ba3-87b8-501664b2a5e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:23:01.941437 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018d9921-58d6-4ba3-87b8-501664b2a5e6 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:23:01.941986 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018d9921-58d6-4ba3-87b8-501664b2a5e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:23:01.942263 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018d9921-58d6-4ba3-87b8-501664b2a5e6 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:23:01.953127 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-018d9921-58d6-4ba3-87b8-501664b2a5e6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 05:23:01.961569 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 7/13] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jul 25 05:23:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 25 05:23:01.964944 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application [None req-018d9921-58d6-4ba3-87b8-501664b2a5e6 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 05:23:01.964944 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:23:01.964944 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:23:01.964944 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:23:01.964944 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:23:01.964944 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:23:01.964944 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:23:01.964944 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:23:01.964944 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:23:01.964944 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:23:01.964944 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:23:01.964944 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:23:01.964944 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 05:23:01.964944 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 05:23:01.964944 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 05:23:01.964944 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 05:23:01.965847 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 05:23:01.965847 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 05:23:01.965847 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 05:23:01.965847 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 05:23:01.965847 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:23:01.965847 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:23:01.965847 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:23:01.965847 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:23:01.965847 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:23:01.965847 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:23:01.965847 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:23:01.965847 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application with Lock( Jul 25 05:23:01.965847 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:23:01.965847 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:23:01.965847 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:23:01.965847 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:23:01.966677 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:23:01.966677 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:23:01.966677 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:23:01.966677 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:23:01.966677 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 05:23:01.966677 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 05:23:01.966677 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 05:23:01.966677 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 05:23:01.966677 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 05:23:01.966677 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 05:23:01.966677 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 05:23:01.966677 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Jul 25 05:23:01.966677 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 7/14] 10.222.0.33 () {56 vars in 1096 bytes} [Mon Jul 25 05:23:01 2022] GET /identity/v3/roles/admin => generated 83 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 25 05:23:01.972627 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5edb81a7-d4f7-4de3-8e41-6e1679608cec None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:23:01.984026 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-018d9921-58d6-4ba3-87b8-501664b2a5e6 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:23:02.064390 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:23:02.073155 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5edb81a7-d4f7-4de3-8e41-6e1679608cec None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:23:02.075504 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:23:02.083153 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-018d9921-58d6-4ba3-87b8-501664b2a5e6 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:23:02.113644 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b3fe721-2ac3-4de1-a570-31de2f5e7338 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:23:02.114568 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3fe721-2ac3-4de1-a570-31de2f5e7338 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:23:02.114860 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3fe721-2ac3-4de1-a570-31de2f5e7338 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:23:02.115143 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3fe721-2ac3-4de1-a570-31de2f5e7338 None admin] PATH_INFO: `/v3/roles` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:23:02.116068 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b3fe721-2ac3-4de1-a570-31de2f5e7338 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:23:02.116379 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b3fe721-2ac3-4de1-a570-31de2f5e7338 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:23:02.117847 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b3fe721-2ac3-4de1-a570-31de2f5e7338 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:23:02.117942 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db00f4eb-5fa0-41e0-968a-98511e5d7008 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:23:02.118496 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db00f4eb-5fa0-41e0-968a-98511e5d7008 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:23:02.118779 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db00f4eb-5fa0-41e0-968a-98511e5d7008 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:23:02.119055 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db00f4eb-5fa0-41e0-968a-98511e5d7008 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:23:02.129585 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 8/15] 10.222.0.33 () {56 vars in 1099 bytes} [Mon Jul 25 05:23:01 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 05:23:02.135107 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b3fe721-2ac3-4de1-a570-31de2f5e7338 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:23:02.188758 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db00f4eb-5fa0-41e0-968a-98511e5d7008 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:23:02.190307 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db00f4eb-5fa0-41e0-968a-98511e5d7008 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:23:02.234697 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b3fe721-2ac3-4de1-a570-31de2f5e7338 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:23:02.268259 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c2b8802-6943-40ce-a499-d29a423029be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:23:02.269163 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2b8802-6943-40ce-a499-d29a423029be None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:23:02.269443 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2b8802-6943-40ce-a499-d29a423029be None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:23:02.269731 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2b8802-6943-40ce-a499-d29a423029be None admin] PATH_INFO: `/v3/users/1eae540242d94c10b0ad33960a855f08` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:23:02.269854 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:23:02.271300 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c2b8802-6943-40ce-a499-d29a423029be None admin] RBAC: Authorizing `identity:get_user(user_id=1eae540242d94c10b0ad33960a855f08)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:23:02.272805 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c2b8802-6943-40ce-a499-d29a423029be None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:23:02.274469 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 9/16] 10.222.0.33 () {56 vars in 1177 bytes} [Mon Jul 25 05:23:02 2022] GET /identity/v3/users/1eae540242d94c10b0ad33960a855f08 => generated 247 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 05:23:02.279779 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c2b8802-6943-40ce-a499-d29a423029be None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:23:02.289283 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 8/17] 10.222.0.33 () {58 vars in 1282 bytes} [Mon Jul 25 05:23:01 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 25 05:23:02.295217 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db00f4eb-5fa0-41e0-968a-98511e5d7008 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:23:02.313613 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:23:02.379260 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c2b8802-6943-40ce-a499-d29a423029be None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:23:02.391829 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db00f4eb-5fa0-41e0-968a-98511e5d7008 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:23:02.412260 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c765b8-77f1-4d5d-b08e-35966e3a5218 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:23:02.413084 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c765b8-77f1-4d5d-b08e-35966e3a5218 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:23:02.413372 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c765b8-77f1-4d5d-b08e-35966e3a5218 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:23:02.413652 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c765b8-77f1-4d5d-b08e-35966e3a5218 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:23:02.414859 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46c765b8-77f1-4d5d-b08e-35966e3a5218 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:23:02.416312 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46c765b8-77f1-4d5d-b08e-35966e3a5218 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:23:02.417952 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 10/18] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Jul 25 05:23:02 2022] GET /identity/v3/domains/default => generated 199 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:23:02.423611 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c765b8-77f1-4d5d-b08e-35966e3a5218 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:23:02.425898 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c9fd4f1-6d73-4943-a5df-af71d78232b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:23:02.426695 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9fd4f1-6d73-4943-a5df-af71d78232b5 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:23:02.426982 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9fd4f1-6d73-4943-a5df-af71d78232b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:23:02.427257 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9fd4f1-6d73-4943-a5df-af71d78232b5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:23:02.437673 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c9fd4f1-6d73-4943-a5df-af71d78232b5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:23:02.439134 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c9fd4f1-6d73-4943-a5df-af71d78232b5 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:23:02.457805 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application [None req-0c9fd4f1-6d73-4943-a5df-af71d78232b5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 05:23:02.457805 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:23:02.457805 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:23:02.457805 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:23:02.457805 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:23:02.457805 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:23:02.457805 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:23:02.457805 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:23:02.457805 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:23:02.457805 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:23:02.457805 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:23:02.457805 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:23:02.457805 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 05:23:02.457805 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 05:23:02.457805 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 05:23:02.458779 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 05:23:02.458779 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:23:02.458779 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:23:02.458779 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:23:02.458779 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:23:02.458779 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:23:02.458779 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:23:02.458779 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:23:02.458779 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application with Lock( Jul 25 05:23:02.458779 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:23:02.458779 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:23:02.458779 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:23:02.458779 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:23:02.458779 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:23:02.458779 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:23:02.458779 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:23:02.458779 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:23:02.459589 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 05:23:02.459589 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 05:23:02.459589 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 05:23:02.459589 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Jul 25 05:23:02.459589 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 9/19] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Jul 25 05:23:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:23:02.464710 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c9fd4f1-6d73-4943-a5df-af71d78232b5 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:23:02.498857 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:23:02.525858 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c765b8-77f1-4d5d-b08e-35966e3a5218 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:23:02.527640 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:23:02.560459 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc911a3-576d-4b88-83e8-9d5b0ab5601f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:23:02.561144 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc911a3-576d-4b88-83e8-9d5b0ab5601f None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:23:02.561437 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc911a3-576d-4b88-83e8-9d5b0ab5601f None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:23:02.561722 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc911a3-576d-4b88-83e8-9d5b0ab5601f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:23:02.562573 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfc911a3-576d-4b88-83e8-9d5b0ab5601f None admin] RBAC: Adding query filter params (role.id=39e5c666285a42fe8fc2c3afca6260d1, scope.domain.id=default, user.id=1eae540242d94c10b0ad33960a855f08) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:23:02.562890 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfc911a3-576d-4b88-83e8-9d5b0ab5601f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:23:02.564344 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfc911a3-576d-4b88-83e8-9d5b0ab5601f None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:23:02.565655 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c9fd4f1-6d73-4943-a5df-af71d78232b5 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:23:02.573979 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 11/20] 10.222.0.33 () {56 vars in 1322 bytes} [Mon Jul 25 05:23:02 2022] GET /identity/v3/role_assignments?user.id=1eae540242d94c10b0ad33960a855f08&scope.domain.id=default&role.id=39e5c666285a42fe8fc2c3afca6260d1 => generated 233 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 05:23:02.601145 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a87964d6-00e5-4de5-9e60-ab575bf99fbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:23:02.601838 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87964d6-00e5-4de5-9e60-ab575bf99fbc None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:23:02.602131 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87964d6-00e5-4de5-9e60-ab575bf99fbc None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:23:02.602414 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87964d6-00e5-4de5-9e60-ab575bf99fbc None admin] PATH_INFO: `/v3/domains` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:23:02.603177 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a87964d6-00e5-4de5-9e60-ab575bf99fbc None admin] RBAC: Adding query filter params (name=Default) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:23:02.603492 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a87964d6-00e5-4de5-9e60-ab575bf99fbc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:23:02.604963 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a87964d6-00e5-4de5-9e60-ab575bf99fbc None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:23:02.625264 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 10/21] 10.222.0.33 () {56 vars in 1109 bytes} [Mon Jul 25 05:23:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 05:23:04.737614 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2832c6-44a3-490b-8671-923e474b4415 None None] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:23:04.738681 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2832c6-44a3-490b-8671-923e474b4415 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:23:04.739165 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2832c6-44a3-490b-8671-923e474b4415 None None] PATH_INFO: `/` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:23:04.740860 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 12/22] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jul 25 05:23:04 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 25 05:23:04.746331 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51e98e8-67c5-4974-8cea-418b8463e846 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:23:04.747177 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51e98e8-67c5-4974-8cea-418b8463e846 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:23:04.747664 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51e98e8-67c5-4974-8cea-418b8463e846 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:23:04.773892 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-e51e98e8-67c5-4974-8cea-418b8463e846 None None] MFA Rules not processed for user `1eae540242d94c10b0ad33960a855f08`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:23:04.775125 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-e51e98e8-67c5-4974-8cea-418b8463e846 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:23:04.864343 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 11/23] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jul 25 05:23:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 05:23:04.896369 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce2832c6-44a3-490b-8671-923e474b4415 None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:23:05.047560 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce2832c6-44a3-490b-8671-923e474b4415 None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:23:05.098381 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6fa63b3-698b-44ae-82c5-722905b5c59d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:23:05.099187 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6fa63b3-698b-44ae-82c5-722905b5c59d None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:23:05.099466 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6fa63b3-698b-44ae-82c5-722905b5c59d None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:23:05.099735 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6fa63b3-698b-44ae-82c5-722905b5c59d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:23:05.112978 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6fa63b3-698b-44ae-82c5-722905b5c59d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 05:23:05.124076 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application [None req-d6fa63b3-698b-44ae-82c5-722905b5c59d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 05:23:05.124076 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:23:05.124076 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:23:05.124076 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:23:05.124076 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:23:05.124076 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:23:05.124076 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:23:05.124076 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:23:05.124076 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:23:05.124076 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:23:05.124076 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:23:05.124076 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:23:05.124076 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 05:23:05.124076 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 05:23:05.124076 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 05:23:05.124076 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 05:23:05.124939 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 05:23:05.124939 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 05:23:05.124939 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 05:23:05.124939 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 05:23:05.124939 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:23:05.124939 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:23:05.124939 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:23:05.124939 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:23:05.124939 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:23:05.124939 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:23:05.124939 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:23:05.124939 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application with Lock( Jul 25 05:23:05.124939 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:23:05.124939 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:23:05.124939 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:23:05.124939 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:23:05.125733 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:23:05.125733 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:23:05.125733 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:23:05.125733 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:23:05.125733 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 05:23:05.125733 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 05:23:05.125733 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 05:23:05.125733 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 05:23:05.125733 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 05:23:05.125733 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 05:23:05.125733 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 05:23:05.125733 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Jul 25 05:23:05.125733 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 13/24] 10.222.0.33 () {56 vars in 1096 bytes} [Mon Jul 25 05:23:04 2022] GET /identity/v3/roles/admin => generated 83 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:23:05.132055 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e51e98e8-67c5-4974-8cea-418b8463e846 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:23:05.161804 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:23:05.255236 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e51e98e8-67c5-4974-8cea-418b8463e846 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:23:05.257164 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816533c2-3c5b-4607-a981-cbf294af621a None None] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:23:05.257454 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816533c2-3c5b-4607-a981-cbf294af621a None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:23:05.257732 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816533c2-3c5b-4607-a981-cbf294af621a None None] PATH_INFO: `/` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:23:05.259261 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 14/25] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jul 25 05:23:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:23:05.264026 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b8fb47-4220-4964-a45b-4a5516486aa8 None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:23:05.264308 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b8fb47-4220-4964-a45b-4a5516486aa8 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:23:05.264586 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b8fb47-4220-4964-a45b-4a5516486aa8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:23:05.291869 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-14b8fb47-4220-4964-a45b-4a5516486aa8 None None] MFA Rules not processed for user `1eae540242d94c10b0ad33960a855f08`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:23:05.293094 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-14b8fb47-4220-4964-a45b-4a5516486aa8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:23:05.329592 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a04d0648-7496-422c-8bdd-65f4b8f23d4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:23:05.330256 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04d0648-7496-422c-8bdd-65f4b8f23d4c None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:23:05.330547 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04d0648-7496-422c-8bdd-65f4b8f23d4c None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:23:05.330820 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04d0648-7496-422c-8bdd-65f4b8f23d4c None admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:23:05.331585 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a04d0648-7496-422c-8bdd-65f4b8f23d4c None admin] RBAC: Adding query filter params (name=admin) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:23:05.331894 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a04d0648-7496-422c-8bdd-65f4b8f23d4c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:23:05.333334 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a04d0648-7496-422c-8bdd-65f4b8f23d4c None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:23:05.337083 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:23:05.351017 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 12/26] 10.222.0.33 () {56 vars in 1099 bytes} [Mon Jul 25 05:23:05 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:23:05.356747 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a04d0648-7496-422c-8bdd-65f4b8f23d4c None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:23:05.434390 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 15/27] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jul 25 05:23:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 05:23:05.450511 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:23:05.472499 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a04d0648-7496-422c-8bdd-65f4b8f23d4c None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:23:05.479475 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b8fb47-4220-4964-a45b-4a5516486aa8 None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:23:05.526272 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb14925-4c36-4500-9cb2-6d3b974549a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:23:05.527160 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb14925-4c36-4500-9cb2-6d3b974549a2 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:23:05.527441 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb14925-4c36-4500-9cb2-6d3b974549a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:23:05.527713 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb14925-4c36-4500-9cb2-6d3b974549a2 None admin] PATH_INFO: `/v3/users/1eae540242d94c10b0ad33960a855f08` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:23:05.540805 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcb14925-4c36-4500-9cb2-6d3b974549a2 None admin] RBAC: Authorizing `identity:get_user(user_id=1eae540242d94c10b0ad33960a855f08)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:23:05.540805 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcb14925-4c36-4500-9cb2-6d3b974549a2 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:23:05.540805 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 13/28] 10.222.0.33 () {56 vars in 1177 bytes} [Mon Jul 25 05:23:05 2022] GET /identity/v3/users/1eae540242d94c10b0ad33960a855f08 => generated 247 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 05:23:05.543363 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb14925-4c36-4500-9cb2-6d3b974549a2 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:23:05.687498 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b8fb47-4220-4964-a45b-4a5516486aa8 None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:23:05.712993 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb14925-4c36-4500-9cb2-6d3b974549a2 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:23:05.733681 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4738f48-f02d-4fb3-a541-1df138d31bd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:23:05.735216 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4738f48-f02d-4fb3-a541-1df138d31bd2 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:23:05.735792 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4738f48-f02d-4fb3-a541-1df138d31bd2 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:23:05.736340 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4738f48-f02d-4fb3-a541-1df138d31bd2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:23:05.749253 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f75693-d271-4ecb-8d18-e47f8330e176 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:23:05.750549 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f75693-d271-4ecb-8d18-e47f8330e176 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:23:05.750993 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f75693-d271-4ecb-8d18-e47f8330e176 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:23:05.751407 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f75693-d271-4ecb-8d18-e47f8330e176 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:23:05.753365 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53f75693-d271-4ecb-8d18-e47f8330e176 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:23:05.754951 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53f75693-d271-4ecb-8d18-e47f8330e176 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:23:05.756913 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 14/29] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Jul 25 05:23:05 2022] GET /identity/v3/domains/default => generated 199 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 05:23:05.758336 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa21099b-f935-4369-ac8e-0b5ddb8f8a9b None None] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:23:05.758762 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa21099b-f935-4369-ac8e-0b5ddb8f8a9b None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:23:05.759260 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa21099b-f935-4369-ac8e-0b5ddb8f8a9b None None] PATH_INFO: `/` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:23:05.760614 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 15/30] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jul 25 05:23:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:23:05.761860 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65828af-3395-4b21-b18e-e08f9f63bb79 None None] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:23:05.762385 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65828af-3395-4b21-b18e-e08f9f63bb79 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:23:05.762803 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65828af-3395-4b21-b18e-e08f9f63bb79 None None] PATH_INFO: `/` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:23:05.764254 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 16/31] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jul 25 05:23:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:23:05.765193 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d65828af-3395-4b21-b18e-e08f9f63bb79 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:23:05.766017 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4738f48-f02d-4fb3-a541-1df138d31bd2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:23:05.767874 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4738f48-f02d-4fb3-a541-1df138d31bd2 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:23:05.792078 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:23:05.793805 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application [None req-b4738f48-f02d-4fb3-a541-1df138d31bd2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 05:23:05.793805 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:23:05.793805 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:23:05.793805 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:23:05.793805 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:23:05.793805 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:23:05.793805 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:23:05.793805 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:23:05.793805 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:23:05.793805 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:23:05.793805 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:23:05.793805 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:23:05.793805 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 05:23:05.793805 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 05:23:05.793805 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 05:23:05.794624 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 05:23:05.794624 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:23:05.794624 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:23:05.794624 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:23:05.794624 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:23:05.794624 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:23:05.794624 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:23:05.794624 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:23:05.794624 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application with Lock( Jul 25 05:23:05.794624 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:23:05.794624 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:23:05.794624 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:23:05.794624 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:23:05.794624 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:23:05.794624 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:23:05.794624 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:23:05.794624 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:23:05.795435 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 05:23:05.795435 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 05:23:05.795435 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 05:23:05.795435 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Jul 25 05:23:05.795435 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 16/32] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Jul 25 05:23:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 316 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 25 05:23:05.805412 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e94dac-5ce9-4a5a-b21c-0859b36d5ad8 None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:23:05.805711 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e94dac-5ce9-4a5a-b21c-0859b36d5ad8 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:23:05.806112 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e94dac-5ce9-4a5a-b21c-0859b36d5ad8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:23:05.828750 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-13e94dac-5ce9-4a5a-b21c-0859b36d5ad8 None None] MFA Rules not processed for user `1eae540242d94c10b0ad33960a855f08`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:23:05.831324 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-13e94dac-5ce9-4a5a-b21c-0859b36d5ad8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:23:05.845138 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:23:05.898047 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d65828af-3395-4b21-b18e-e08f9f63bb79 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:23:05.913572 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 17/33] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jul 25 05:23:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 05:23:05.915340 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd2f280-3a93-4ea3-ade2-469b88ae53bc None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:23:05.915806 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd2f280-3a93-4ea3-ade2-469b88ae53bc None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:23:05.916272 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd2f280-3a93-4ea3-ade2-469b88ae53bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:23:05.932637 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-cbd2f280-3a93-4ea3-ade2-469b88ae53bc None None] MFA Rules not processed for user `1eae540242d94c10b0ad33960a855f08`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:23:05.934125 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-cbd2f280-3a93-4ea3-ade2-469b88ae53bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:23:05.938224 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d5cb676-c2ef-4668-8b86-3aa3b7738cad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:23:05.939640 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5cb676-c2ef-4668-8b86-3aa3b7738cad None admin] REQUEST_METHOD: `PUT` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:23:05.939983 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5cb676-c2ef-4668-8b86-3aa3b7738cad None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:23:05.940263 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5cb676-c2ef-4668-8b86-3aa3b7738cad None admin] PATH_INFO: `/v3/domains/default/users/1eae540242d94c10b0ad33960a855f08/roles/39e5c666285a42fe8fc2c3afca6260d1` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:23:05.942808 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d5cb676-c2ef-4668-8b86-3aa3b7738cad None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=1eae540242d94c10b0ad33960a855f08, role_id=39e5c666285a42fe8fc2c3afca6260d1)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:23:05.944472 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d5cb676-c2ef-4668-8b86-3aa3b7738cad None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:23:06.012175 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 18/34] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jul 25 05:23:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 05:23:06.013597 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd2f280-3a93-4ea3-ade2-469b88ae53bc None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:23:06.090629 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:23:06.091715 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 17/35] 10.222.0.33 () {58 vars in 1361 bytes} [Mon Jul 25 05:23:05 2022] PUT /identity/v3/domains/default/users/1eae540242d94c10b0ad33960a855f08/roles/39e5c666285a42fe8fc2c3afca6260d1 => generated 0 bytes in 327 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 05:23:06.093557 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d5cb676-c2ef-4668-8b86-3aa3b7738cad None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:23:06.133724 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd2f280-3a93-4ea3-ade2-469b88ae53bc None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:23:06.178108 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de091b21-a2db-49d4-ab64-bd656d7bdc32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:23:06.179081 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de091b21-a2db-49d4-ab64-bd656d7bdc32 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:23:06.179528 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de091b21-a2db-49d4-ab64-bd656d7bdc32 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:23:06.179944 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de091b21-a2db-49d4-ab64-bd656d7bdc32 None admin] PATH_INFO: `/v3/domains` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:23:06.180867 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de091b21-a2db-49d4-ab64-bd656d7bdc32 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:23:06.181385 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de091b21-a2db-49d4-ab64-bd656d7bdc32 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:23:06.183010 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de091b21-a2db-49d4-ab64-bd656d7bdc32 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:23:06.205308 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 19/36] 10.222.0.33 () {56 vars in 1109 bytes} [Mon Jul 25 05:23:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:23:06.206587 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de091b21-a2db-49d4-ab64-bd656d7bdc32 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:23:06.321978 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de091b21-a2db-49d4-ab64-bd656d7bdc32 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:23:06.353786 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:23:06.364077 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad59e443-ea50-4b07-b0f9-96cee0c10ae9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:23:06.365265 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad59e443-ea50-4b07-b0f9-96cee0c10ae9 None admin] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:23:06.365790 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad59e443-ea50-4b07-b0f9-96cee0c10ae9 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:23:06.366259 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad59e443-ea50-4b07-b0f9-96cee0c10ae9 None admin] PATH_INFO: `/v3/roles` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:23:06.367252 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad59e443-ea50-4b07-b0f9-96cee0c10ae9 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:23:06.369444 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad59e443-ea50-4b07-b0f9-96cee0c10ae9 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:23:06.412706 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:23:06.438295 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d5cb676-c2ef-4668-8b86-3aa3b7738cad None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:23:06.485011 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce80b5c-99e1-414e-8fd4-bb032c95d4e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:23:06.486139 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce80b5c-99e1-414e-8fd4-bb032c95d4e0 None admin] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:23:06.486619 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce80b5c-99e1-414e-8fd4-bb032c95d4e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:23:06.487125 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce80b5c-99e1-414e-8fd4-bb032c95d4e0 None admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:23:06.488131 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ce80b5c-99e1-414e-8fd4-bb032c95d4e0 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:23:06.490142 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ce80b5c-99e1-414e-8fd4-bb032c95d4e0 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:23:06.516396 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 20/37] 10.222.0.33 () {60 vars in 1131 bytes} [Mon Jul 25 05:23:06 2022] POST /identity/v3/roles => generated 219 bytes in 310 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 05:23:06.519266 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad59e443-ea50-4b07-b0f9-96cee0c10ae9 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:23:06.528094 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 18/38] 10.222.0.33 () {60 vars in 1131 bytes} [Mon Jul 25 05:23:06 2022] POST /identity/v3/roles => generated 225 bytes in 436 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 05:23:06.602519 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7221571f-190f-44da-b959-a56e9540f9b9 None None] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:23:06.603249 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7221571f-190f-44da-b959-a56e9540f9b9 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:23:06.603756 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7221571f-190f-44da-b959-a56e9540f9b9 None None] PATH_INFO: `/` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:23:06.605506 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 19/39] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jul 25 05:23:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 25 05:23:06.620713 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48361eef-8c6f-41f0-8cb1-38a39c7f1262 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:23:06.621386 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48361eef-8c6f-41f0-8cb1-38a39c7f1262 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:23:06.621878 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48361eef-8c6f-41f0-8cb1-38a39c7f1262 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:23:06.656519 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-48361eef-8c6f-41f0-8cb1-38a39c7f1262 None None] MFA Rules not processed for user `1eae540242d94c10b0ad33960a855f08`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:23:06.659034 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-48361eef-8c6f-41f0-8cb1-38a39c7f1262 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:23:06.711656 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad59e443-ea50-4b07-b0f9-96cee0c10ae9 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:23:06.766597 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c2ce86b-93f9-4fe6-bd00-dac936683160 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:23:06.768247 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2ce86b-93f9-4fe6-bd00-dac936683160 None admin] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:23:06.769433 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2ce86b-93f9-4fe6-bd00-dac936683160 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:23:06.769988 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2ce86b-93f9-4fe6-bd00-dac936683160 None admin] PATH_INFO: `/v3/projects` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:23:06.771240 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c2ce86b-93f9-4fe6-bd00-dac936683160 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:23:06.773942 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c2ce86b-93f9-4fe6-bd00-dac936683160 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:23:06.791389 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 20/40] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jul 25 05:23:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 363 bytes (3 switches on core 0) Jul 25 05:23:06.792700 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0d5be9-6d00-4dc7-a055-0457c3ed8793 None None] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:23:06.793193 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0d5be9-6d00-4dc7-a055-0457c3ed8793 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:23:06.793482 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0d5be9-6d00-4dc7-a055-0457c3ed8793 None None] PATH_INFO: `/` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:23:06.794984 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 21/41] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jul 25 05:23:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:23:06.802900 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796574a2-9d11-4bff-a38e-8c447d8ad2e6 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:23:06.802900 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796574a2-9d11-4bff-a38e-8c447d8ad2e6 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:23:06.803140 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796574a2-9d11-4bff-a38e-8c447d8ad2e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:23:06.814727 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 21/42] 10.222.0.33 () {60 vars in 1141 bytes} [Mon Jul 25 05:23:06 2022] POST /identity/v3/projects => generated 301 bytes in 296 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 05:23:06.827223 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-796574a2-9d11-4bff-a38e-8c447d8ad2e6 None None] MFA Rules not processed for user `1eae540242d94c10b0ad33960a855f08`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:23:06.828749 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c2ce86b-93f9-4fe6-bd00-dac936683160 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:23:06.828907 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-796574a2-9d11-4bff-a38e-8c447d8ad2e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:23:06.896142 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:23:06.934380 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 22/43] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jul 25 05:23:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 25 05:23:06.961392 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-796574a2-9d11-4bff-a38e-8c447d8ad2e6 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:23:06.963963 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:23:06.986931 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c2ce86b-93f9-4fe6-bd00-dac936683160 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:23:07.035033 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14365b70-b8cf-49b5-bf4a-e5f4ceb66b49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:23:07.038936 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14365b70-b8cf-49b5-bf4a-e5f4ceb66b49 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:23:07.038936 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14365b70-b8cf-49b5-bf4a-e5f4ceb66b49 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:23:07.038936 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14365b70-b8cf-49b5-bf4a-e5f4ceb66b49 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:23:07.039666 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14365b70-b8cf-49b5-bf4a-e5f4ceb66b49 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:23:07.041169 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14365b70-b8cf-49b5-bf4a-e5f4ceb66b49 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:23:07.042859 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 22/44] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Jul 25 05:23:06 2022] GET /identity/v3/domains/default => generated 199 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:23:07.049715 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14365b70-b8cf-49b5-bf4a-e5f4ceb66b49 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:23:07.085150 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-796574a2-9d11-4bff-a38e-8c447d8ad2e6 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:23:07.119572 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05187095-72da-4b68-8ec2-cc3e4b7c0316 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:23:07.120248 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05187095-72da-4b68-8ec2-cc3e4b7c0316 None admin] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:23:07.120538 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05187095-72da-4b68-8ec2-cc3e4b7c0316 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:23:07.120836 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05187095-72da-4b68-8ec2-cc3e4b7c0316 None admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:23:07.121603 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05187095-72da-4b68-8ec2-cc3e4b7c0316 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:23:07.122994 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05187095-72da-4b68-8ec2-cc3e4b7c0316 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:23:07.138639 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 23/45] 10.222.0.33 () {60 vars in 1131 bytes} [Mon Jul 25 05:23:06 2022] POST /identity/v3/roles => generated 223 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 05:23:07.160452 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14365b70-b8cf-49b5-bf4a-e5f4ceb66b49 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:23:07.171965 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:23:07.202063 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21741419-992e-4558-8310-341d13c9ff1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:23:07.202755 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21741419-992e-4558-8310-341d13c9ff1c None admin] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:23:07.203045 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21741419-992e-4558-8310-341d13c9ff1c None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:23:07.203323 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21741419-992e-4558-8310-341d13c9ff1c None admin] PATH_INFO: `/v3/projects` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:23:07.204057 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21741419-992e-4558-8310-341d13c9ff1c None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:23:07.205646 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21741419-992e-4558-8310-341d13c9ff1c None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:23:07.240831 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 23/46] 10.222.0.33 () {60 vars in 1141 bytes} [Mon Jul 25 05:23:07 2022] POST /identity/v3/projects => generated 312 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 25 05:23:08.543269 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94982243-ead7-4634-b109-1c9deadd4e35 None None] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:23:08.543802 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94982243-ead7-4634-b109-1c9deadd4e35 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:23:08.543886 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94982243-ead7-4634-b109-1c9deadd4e35 None None] PATH_INFO: `/` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:23:08.545278 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 24/47] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jul 25 05:23:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:23:08.551258 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4eebd9-1ac1-4ef3-b67b-6cf4c0b7f231 None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:23:08.551650 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4eebd9-1ac1-4ef3-b67b-6cf4c0b7f231 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:23:08.551827 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4eebd9-1ac1-4ef3-b67b-6cf4c0b7f231 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:23:08.569362 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-0a4eebd9-1ac1-4ef3-b67b-6cf4c0b7f231 None None] MFA Rules not processed for user `1eae540242d94c10b0ad33960a855f08`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:23:08.570882 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-0a4eebd9-1ac1-4ef3-b67b-6cf4c0b7f231 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:23:08.635559 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 24/48] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jul 25 05:23:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 05:23:08.658316 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94982243-ead7-4634-b109-1c9deadd4e35 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:23:08.721644 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:23:08.761947 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94982243-ead7-4634-b109-1c9deadd4e35 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:23:08.795434 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2757a2f-abf4-4f14-8a5d-4c43f6a0499b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:23:08.796164 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2757a2f-abf4-4f14-8a5d-4c43f6a0499b None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:23:08.796454 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2757a2f-abf4-4f14-8a5d-4c43f6a0499b None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:23:08.796733 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2757a2f-abf4-4f14-8a5d-4c43f6a0499b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:23:08.808141 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2757a2f-abf4-4f14-8a5d-4c43f6a0499b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 05:23:08.816932 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application [None req-d2757a2f-abf4-4f14-8a5d-4c43f6a0499b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 05:23:08.816932 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:23:08.816932 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:23:08.816932 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:23:08.816932 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:23:08.816932 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:23:08.816932 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:23:08.816932 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:23:08.816932 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:23:08.816932 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:23:08.816932 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:23:08.816932 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:23:08.816932 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 05:23:08.816932 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 05:23:08.816932 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 05:23:08.816932 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 05:23:08.817836 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 05:23:08.817836 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 05:23:08.817836 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 05:23:08.817836 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 05:23:08.817836 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:23:08.817836 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:23:08.817836 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:23:08.817836 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:23:08.817836 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:23:08.817836 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:23:08.817836 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:23:08.817836 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application with Lock( Jul 25 05:23:08.817836 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:23:08.817836 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:23:08.817836 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:23:08.817836 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:23:08.818632 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:23:08.818632 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:23:08.818632 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:23:08.818632 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:23:08.818632 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 05:23:08.818632 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 05:23:08.818632 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 05:23:08.818632 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 05:23:08.818632 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 05:23:08.818632 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 05:23:08.818632 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 05:23:08.818632 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Jul 25 05:23:08.818632 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 25/49] 10.222.0.33 () {56 vars in 1096 bytes} [Mon Jul 25 05:23:08 2022] GET /identity/v3/roles/admin => generated 83 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:23:08.823997 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a4eebd9-1ac1-4ef3-b67b-6cf4c0b7f231 None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:23:08.914324 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:23:08.926552 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a4eebd9-1ac1-4ef3-b67b-6cf4c0b7f231 None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:23:08.961436 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b672858-e9ed-4984-957e-84144db5b867 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:23:08.962116 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b672858-e9ed-4984-957e-84144db5b867 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:23:08.962404 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b672858-e9ed-4984-957e-84144db5b867 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:23:08.962687 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b672858-e9ed-4984-957e-84144db5b867 None admin] PATH_INFO: `/v3/roles` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:23:08.963447 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b672858-e9ed-4984-957e-84144db5b867 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:23:08.963759 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b672858-e9ed-4984-957e-84144db5b867 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:23:08.965176 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b672858-e9ed-4984-957e-84144db5b867 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:23:08.974239 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 25/50] 10.222.0.33 () {56 vars in 1099 bytes} [Mon Jul 25 05:23:08 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:23:08.979735 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2757a2f-abf4-4f14-8a5d-4c43f6a0499b None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:23:09.080126 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2757a2f-abf4-4f14-8a5d-4c43f6a0499b None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:23:09.101936 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:23:09.113835 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc52440-4c7d-426b-b858-3a34135943c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:23:09.114715 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc52440-4c7d-426b-b858-3a34135943c5 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:23:09.115007 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc52440-4c7d-426b-b858-3a34135943c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:23:09.115291 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc52440-4c7d-426b-b858-3a34135943c5 None admin] PATH_INFO: `/v3/users/1eae540242d94c10b0ad33960a855f08` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:23:09.116510 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bc52440-4c7d-426b-b858-3a34135943c5 None admin] RBAC: Authorizing `identity:get_user(user_id=1eae540242d94c10b0ad33960a855f08)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:23:09.118009 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bc52440-4c7d-426b-b858-3a34135943c5 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:23:09.119660 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 26/51] 10.222.0.33 () {56 vars in 1177 bytes} [Mon Jul 25 05:23:08 2022] GET /identity/v3/users/1eae540242d94c10b0ad33960a855f08 => generated 247 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:23:09.125127 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b672858-e9ed-4984-957e-84144db5b867 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:23:09.280134 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1adea48b-6d84-49fd-9b5f-a9acf48032e7 None None] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:23:09.280423 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1adea48b-6d84-49fd-9b5f-a9acf48032e7 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:23:09.280705 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1adea48b-6d84-49fd-9b5f-a9acf48032e7 None None] PATH_INFO: `/` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:23:09.282129 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 27/52] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jul 25 05:23:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 25 05:23:09.287040 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3d92d5-639f-40ea-8384-93779c5a3e85 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:23:09.287335 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3d92d5-639f-40ea-8384-93779c5a3e85 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:23:09.287622 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3d92d5-639f-40ea-8384-93779c5a3e85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:23:09.304330 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-4b3d92d5-639f-40ea-8384-93779c5a3e85 None None] MFA Rules not processed for user `1eae540242d94c10b0ad33960a855f08`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:23:09.305553 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-4b3d92d5-639f-40ea-8384-93779c5a3e85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:23:09.330061 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b672858-e9ed-4984-957e-84144db5b867 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:23:09.362847 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ebbb81-0fb1-46a5-bd6a-f0d41ce7b72e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:23:09.363576 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ebbb81-0fb1-46a5-bd6a-f0d41ce7b72e None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:23:09.363864 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ebbb81-0fb1-46a5-bd6a-f0d41ce7b72e None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:23:09.364142 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ebbb81-0fb1-46a5-bd6a-f0d41ce7b72e None admin] PATH_INFO: `/v3/domains/default` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:23:09.365359 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05ebbb81-0fb1-46a5-bd6a-f0d41ce7b72e None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:23:09.366802 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05ebbb81-0fb1-46a5-bd6a-f0d41ce7b72e None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:23:09.368442 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 26/53] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Jul 25 05:23:09 2022] GET /identity/v3/domains/default => generated 199 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:23:09.371373 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 28/54] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jul 25 05:23:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 25 05:23:09.374255 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ebbb81-0fb1-46a5-bd6a-f0d41ce7b72e None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:23:09.395209 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b3d92d5-639f-40ea-8384-93779c5a3e85 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:23:09.395788 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:23:09.472375 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ebbb81-0fb1-46a5-bd6a-f0d41ce7b72e None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:23:09.491243 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b3d92d5-639f-40ea-8384-93779c5a3e85 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:23:09.505798 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-394f91ac-7ea9-4cc6-873c-8dea4bfce5f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:23:09.506454 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394f91ac-7ea9-4cc6-873c-8dea4bfce5f6 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:23:09.506740 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394f91ac-7ea9-4cc6-873c-8dea4bfce5f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:23:09.507024 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394f91ac-7ea9-4cc6-873c-8dea4bfce5f6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:23:09.507893 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-394f91ac-7ea9-4cc6-873c-8dea4bfce5f6 None admin] RBAC: Adding query filter params (role.id=39e5c666285a42fe8fc2c3afca6260d1, scope.domain.id=default, user.id=1eae540242d94c10b0ad33960a855f08) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:23:09.508212 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-394f91ac-7ea9-4cc6-873c-8dea4bfce5f6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:23:09.509670 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-394f91ac-7ea9-4cc6-873c-8dea4bfce5f6 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:23:09.510964 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:23:09.515674 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 27/55] 10.222.0.33 () {56 vars in 1322 bytes} [Mon Jul 25 05:23:09 2022] GET /identity/v3/role_assignments?user.id=1eae540242d94c10b0ad33960a855f08&scope.domain.id=default&role.id=39e5c666285a42fe8fc2c3afca6260d1 => generated 530 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 05:23:09.525619 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f94e811f-5e26-4b6c-b847-1ec2588545cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:23:09.526453 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94e811f-5e26-4b6c-b847-1ec2588545cb None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:23:09.526744 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94e811f-5e26-4b6c-b847-1ec2588545cb None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:23:09.527025 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94e811f-5e26-4b6c-b847-1ec2588545cb None admin] PATH_INFO: `/v3/domains/default` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:23:09.528313 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f94e811f-5e26-4b6c-b847-1ec2588545cb None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:23:09.529870 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f94e811f-5e26-4b6c-b847-1ec2588545cb None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:23:09.531636 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 29/56] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Jul 25 05:23:09 2022] GET /identity/v3/domains/default => generated 199 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 05:23:09.538212 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-394f91ac-7ea9-4cc6-873c-8dea4bfce5f6 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:23:09.633898 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:23:09.651989 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-394f91ac-7ea9-4cc6-873c-8dea4bfce5f6 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:23:09.688158 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a2191c-a83b-4671-a594-5cd67b14dfa7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:23:09.689078 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a2191c-a83b-4671-a594-5cd67b14dfa7 None admin] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:23:09.689545 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a2191c-a83b-4671-a594-5cd67b14dfa7 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:23:09.689661 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a2191c-a83b-4671-a594-5cd67b14dfa7 None admin] PATH_INFO: `/v3/projects` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:23:09.690512 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3a2191c-a83b-4671-a594-5cd67b14dfa7 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:23:09.692214 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3a2191c-a83b-4671-a594-5cd67b14dfa7 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:23:09.719092 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 28/57] 10.222.0.33 () {60 vars in 1140 bytes} [Mon Jul 25 05:23:09 2022] POST /identity/v3/projects => generated 298 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 05:23:11.742370 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0cd3ce-4440-4b4e-8580-7e0d9a35b901 None None] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:23:11.742897 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0cd3ce-4440-4b4e-8580-7e0d9a35b901 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:23:11.743008 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0cd3ce-4440-4b4e-8580-7e0d9a35b901 None None] PATH_INFO: `/` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:23:11.744326 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 30/58] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jul 25 05:23:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:23:11.749461 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559a9b77-255a-4eb3-87d4-1b231c0aafc7 None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:23:11.749835 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559a9b77-255a-4eb3-87d4-1b231c0aafc7 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:23:11.750031 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559a9b77-255a-4eb3-87d4-1b231c0aafc7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:23:11.767223 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-559a9b77-255a-4eb3-87d4-1b231c0aafc7 None None] MFA Rules not processed for user `1eae540242d94c10b0ad33960a855f08`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:23:11.768513 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-559a9b77-255a-4eb3-87d4-1b231c0aafc7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:23:11.833610 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 29/59] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jul 25 05:23:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 05:23:11.857326 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe0cd3ce-4440-4b4e-8580-7e0d9a35b901 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:23:11.961187 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe0cd3ce-4440-4b4e-8580-7e0d9a35b901 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:23:11.993492 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4cebc38-1dc8-42ed-8a7d-2ecea100c60e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:23:11.994226 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4cebc38-1dc8-42ed-8a7d-2ecea100c60e None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:23:11.994517 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4cebc38-1dc8-42ed-8a7d-2ecea100c60e None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:23:11.994799 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4cebc38-1dc8-42ed-8a7d-2ecea100c60e None admin] PATH_INFO: `/v3/domains/default` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:23:11.995980 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4cebc38-1dc8-42ed-8a7d-2ecea100c60e None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:23:11.997481 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4cebc38-1dc8-42ed-8a7d-2ecea100c60e None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:23:11.999103 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 31/60] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Jul 25 05:23:11 2022] GET /identity/v3/domains/default => generated 199 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:23:12.005430 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-559a9b77-255a-4eb3-87d4-1b231c0aafc7 None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:23:12.068463 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:23:12.102448 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-559a9b77-255a-4eb3-87d4-1b231c0aafc7 None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:23:12.135680 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e56e179b-97aa-4471-b4fd-14cc91adb2f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:23:12.136347 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56e179b-97aa-4471-b4fd-14cc91adb2f1 None admin] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:23:12.136636 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56e179b-97aa-4471-b4fd-14cc91adb2f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:23:12.136948 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56e179b-97aa-4471-b4fd-14cc91adb2f1 None admin] PATH_INFO: `/v3/users` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:23:12.137673 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e56e179b-97aa-4471-b4fd-14cc91adb2f1 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:23:12.139127 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e56e179b-97aa-4471-b4fd-14cc91adb2f1 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:23:12.163717 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 30/61] 10.222.0.33 () {60 vars in 1132 bytes} [Mon Jul 25 05:23:12 2022] POST /identity/v3/users => generated 275 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 05:23:15.302080 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01671a2-ba5d-4f24-9d48-3975035c48d7 None None] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:23:15.302614 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01671a2-ba5d-4f24-9d48-3975035c48d7 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:23:15.302691 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01671a2-ba5d-4f24-9d48-3975035c48d7 None None] PATH_INFO: `/` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:23:15.304310 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 32/62] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jul 25 05:23:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:23:15.309779 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77332dbf-2736-4416-96f8-5d4516282098 None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:23:15.310172 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77332dbf-2736-4416-96f8-5d4516282098 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:23:15.310351 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77332dbf-2736-4416-96f8-5d4516282098 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:23:15.333989 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-77332dbf-2736-4416-96f8-5d4516282098 None None] MFA Rules not processed for user `1eae540242d94c10b0ad33960a855f08`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:23:15.335231 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-77332dbf-2736-4416-96f8-5d4516282098 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:23:15.418672 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 31/63] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jul 25 05:23:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 25 05:23:15.455333 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c01671a2-ba5d-4f24-9d48-3975035c48d7 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:23:15.540204 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c72971-29e8-4954-8201-4bcd3dc859bf None None] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:23:15.540726 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c72971-29e8-4954-8201-4bcd3dc859bf None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:23:15.541178 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c72971-29e8-4954-8201-4bcd3dc859bf None None] PATH_INFO: `/` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:23:15.542718 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 32/64] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jul 25 05:23:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:23:15.548109 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d6df80-8884-40bb-b608-30e756c878e9 None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:23:15.548662 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d6df80-8884-40bb-b608-30e756c878e9 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:23:15.549180 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d6df80-8884-40bb-b608-30e756c878e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:23:15.549399 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:23:15.581898 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-15d6df80-8884-40bb-b608-30e756c878e9 None None] MFA Rules not processed for user `1eae540242d94c10b0ad33960a855f08`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:23:15.583326 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-15d6df80-8884-40bb-b608-30e756c878e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:23:15.615142 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c01671a2-ba5d-4f24-9d48-3975035c48d7 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:23:15.642366 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:23:15.672221 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbccdc61-5ea2-41e1-b950-176d133a8702 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:23:15.673199 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbccdc61-5ea2-41e1-b950-176d133a8702 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:23:15.673490 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbccdc61-5ea2-41e1-b950-176d133a8702 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:23:15.673799 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbccdc61-5ea2-41e1-b950-176d133a8702 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:23:15.680999 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 33/65] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jul 25 05:23:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 05:23:15.690316 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbccdc61-5ea2-41e1-b950-176d133a8702 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 05:23:15.698353 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application [None req-cbccdc61-5ea2-41e1-b950-176d133a8702 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 25 05:23:15.698353 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:23:15.698353 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:23:15.698353 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:23:15.698353 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:23:15.698353 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:23:15.698353 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:23:15.698353 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:23:15.698353 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:23:15.698353 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:23:15.698353 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:23:15.698353 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:23:15.698353 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 05:23:15.698353 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 05:23:15.698353 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 05:23:15.698353 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 05:23:15.699407 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 05:23:15.699407 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 05:23:15.699407 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 05:23:15.699407 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 05:23:15.699407 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:23:15.699407 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:23:15.699407 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:23:15.699407 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:23:15.699407 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:23:15.699407 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:23:15.699407 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:23:15.699407 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application with Lock( Jul 25 05:23:15.699407 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:23:15.699407 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:23:15.699407 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:23:15.699407 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:23:15.700207 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:23:15.700207 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:23:15.700207 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:23:15.700207 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:23:15.700207 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 05:23:15.700207 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 05:23:15.700207 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 05:23:15.700207 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 05:23:15.700207 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 05:23:15.700207 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 05:23:15.700207 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 25 05:23:15.700207 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Jul 25 05:23:15.700207 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 33/66] 10.222.0.33 () {56 vars in 1099 bytes} [Mon Jul 25 05:23:15 2022] GET /identity/v3/roles/member => generated 84 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:23:15.706178 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d6df80-8884-40bb-b608-30e756c878e9 None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:23:15.720989 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbccdc61-5ea2-41e1-b950-176d133a8702 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:23:15.846736 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbccdc61-5ea2-41e1-b950-176d133a8702 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:23:15.856122 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d6df80-8884-40bb-b608-30e756c878e9 None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:23:15.863256 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:23:15.881462 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38dc7a5b-1a7c-43bf-8568-a7cb1477fddf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:23:15.882263 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38dc7a5b-1a7c-43bf-8568-a7cb1477fddf None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:23:15.882554 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38dc7a5b-1a7c-43bf-8568-a7cb1477fddf None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:23:15.882837 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38dc7a5b-1a7c-43bf-8568-a7cb1477fddf None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:23:15.889445 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d82792c7-6f28-4197-a854-758805055e89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:23:15.890113 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82792c7-6f28-4197-a854-758805055e89 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:23:15.890401 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82792c7-6f28-4197-a854-758805055e89 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:23:15.890677 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82792c7-6f28-4197-a854-758805055e89 None admin] PATH_INFO: `/v3/roles` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:23:15.891968 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d82792c7-6f28-4197-a854-758805055e89 None admin] RBAC: Adding query filter params (name=member) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:23:15.892414 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d82792c7-6f28-4197-a854-758805055e89 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:23:15.893732 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38dc7a5b-1a7c-43bf-8568-a7cb1477fddf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 05:23:15.893840 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d82792c7-6f28-4197-a854-758805055e89 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:23:15.901429 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application [None req-38dc7a5b-1a7c-43bf-8568-a7cb1477fddf None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 05:23:15.901429 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:23:15.901429 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:23:15.901429 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:23:15.901429 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:23:15.901429 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:23:15.901429 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:23:15.901429 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:23:15.901429 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:23:15.901429 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:23:15.901429 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:23:15.901429 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:23:15.901429 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 05:23:15.901429 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 05:23:15.901429 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 05:23:15.901429 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 05:23:15.902297 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 05:23:15.902297 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 05:23:15.902297 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 05:23:15.902297 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 05:23:15.902297 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:23:15.902297 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:23:15.902297 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:23:15.902297 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:23:15.902297 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:23:15.902297 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:23:15.902297 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:23:15.902297 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application with Lock( Jul 25 05:23:15.902297 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:23:15.902297 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:23:15.902297 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:23:15.902297 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:23:15.903432 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:23:15.903432 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:23:15.903432 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:23:15.903432 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:23:15.903432 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 05:23:15.903432 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 05:23:15.903432 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 05:23:15.903432 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 05:23:15.903432 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 05:23:15.903432 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 05:23:15.903432 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 05:23:15.903432 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Jul 25 05:23:15.904074 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 34/67] 10.222.0.33 () {56 vars in 1101 bytes} [Mon Jul 25 05:23:15 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 05:23:15.904142 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 34/68] 10.222.0.33 () {56 vars in 1096 bytes} [Mon Jul 25 05:23:15 2022] GET /identity/v3/roles/admin => generated 83 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:23:15.904142 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760a3742-f861-4f02-adca-646d5ce65b52 None None] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:23:15.904271 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760a3742-f861-4f02-adca-646d5ce65b52 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:23:15.904411 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96bc48ec-5e94-40f6-90b0-73f8eacba074 None None] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:23:15.904497 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760a3742-f861-4f02-adca-646d5ce65b52 None None] PATH_INFO: `/` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:23:15.904962 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96bc48ec-5e94-40f6-90b0-73f8eacba074 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:23:15.905373 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96bc48ec-5e94-40f6-90b0-73f8eacba074 None None] PATH_INFO: `/` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:23:15.906038 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 35/69] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jul 25 05:23:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:23:15.907683 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 35/70] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jul 25 05:23:15 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:23:15.908097 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a17c0d-0e33-46a3-afc5-4fcda882c85e None None] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:23:15.908398 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a17c0d-0e33-46a3-afc5-4fcda882c85e None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:23:15.908685 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a17c0d-0e33-46a3-afc5-4fcda882c85e None None] PATH_INFO: `/` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:23:15.909124 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96bc48ec-5e94-40f6-90b0-73f8eacba074 None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:23:15.910214 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 36/71] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jul 25 05:23:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:23:15.913277 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398f7917-c3a5-4378-adbf-e6d4dba6671b None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:23:15.913574 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398f7917-c3a5-4378-adbf-e6d4dba6671b None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:23:15.913860 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398f7917-c3a5-4378-adbf-e6d4dba6671b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:23:15.930352 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-398f7917-c3a5-4378-adbf-e6d4dba6671b None None] MFA Rules not processed for user `1eae540242d94c10b0ad33960a855f08`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:23:15.931566 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-398f7917-c3a5-4378-adbf-e6d4dba6671b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:23:15.959520 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:23:15.998643 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 37/72] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jul 25 05:23:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 05:23:15.999669 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-398f7917-c3a5-4378-adbf-e6d4dba6671b None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:23:16.017987 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96bc48ec-5e94-40f6-90b0-73f8eacba074 None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:23:16.052200 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11710304-e1bd-42e3-ae3b-4337b90ba62d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:23:16.053116 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11710304-e1bd-42e3-ae3b-4337b90ba62d None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:23:16.053408 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11710304-e1bd-42e3-ae3b-4337b90ba62d None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:23:16.053689 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11710304-e1bd-42e3-ae3b-4337b90ba62d None admin] PATH_INFO: `/v3/users/c25734c1f07644de9355e54cfd5f65cc` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:23:16.068822 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11710304-e1bd-42e3-ae3b-4337b90ba62d None admin] RBAC: Authorizing `identity:get_user(user_id=c25734c1f07644de9355e54cfd5f65cc)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:23:16.070294 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11710304-e1bd-42e3-ae3b-4337b90ba62d None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:23:16.072157 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 36/73] 10.222.0.33 () {56 vars in 1177 bytes} [Mon Jul 25 05:23:15 2022] GET /identity/v3/users/c25734c1f07644de9355e54cfd5f65cc => generated 275 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:23:16.073214 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3fb6dba-c1e5-4d8c-8d1b-fd21a8a58548 None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:23:16.073510 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3fb6dba-c1e5-4d8c-8d1b-fd21a8a58548 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:23:16.073794 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3fb6dba-c1e5-4d8c-8d1b-fd21a8a58548 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:23:16.090318 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-f3fb6dba-c1e5-4d8c-8d1b-fd21a8a58548 None None] MFA Rules not processed for user `1eae540242d94c10b0ad33960a855f08`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:23:16.091530 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-f3fb6dba-c1e5-4d8c-8d1b-fd21a8a58548 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:23:16.100633 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-398f7917-c3a5-4378-adbf-e6d4dba6671b None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:23:16.107450 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:23:16.134510 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db9105be-8de1-43aa-962b-ba48cbe0c651 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:23:16.135184 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9105be-8de1-43aa-962b-ba48cbe0c651 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:23:16.135477 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9105be-8de1-43aa-962b-ba48cbe0c651 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:23:16.135764 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9105be-8de1-43aa-962b-ba48cbe0c651 None admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:23:16.136532 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db9105be-8de1-43aa-962b-ba48cbe0c651 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:23:16.136876 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db9105be-8de1-43aa-962b-ba48cbe0c651 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:23:16.138271 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db9105be-8de1-43aa-962b-ba48cbe0c651 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:23:16.146980 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 38/74] 10.222.0.33 () {56 vars in 1099 bytes} [Mon Jul 25 05:23:15 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:23:16.148160 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5cae436-4a1d-44bd-9f79-28f3abbd4968 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:23:16.148457 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5cae436-4a1d-44bd-9f79-28f3abbd4968 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:23:16.148745 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5cae436-4a1d-44bd-9f79-28f3abbd4968 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:23:16.161957 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 37/75] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jul 25 05:23:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 05:23:16.162802 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3fb6dba-c1e5-4d8c-8d1b-fd21a8a58548 None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:23:16.164903 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-d5cae436-4a1d-44bd-9f79-28f3abbd4968 None None] MFA Rules not processed for user `1eae540242d94c10b0ad33960a855f08`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:23:16.166101 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-d5cae436-4a1d-44bd-9f79-28f3abbd4968 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:23:16.194299 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:23:16.232081 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 39/76] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jul 25 05:23:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 05:23:16.232963 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5cae436-4a1d-44bd-9f79-28f3abbd4968 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:23:16.267427 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3fb6dba-c1e5-4d8c-8d1b-fd21a8a58548 None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:23:16.299939 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-059799b9-e7c9-4494-8e7f-77e467c8c882 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:23:16.300742 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059799b9-e7c9-4494-8e7f-77e467c8c882 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:23:16.301059 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059799b9-e7c9-4494-8e7f-77e467c8c882 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:23:16.301337 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059799b9-e7c9-4494-8e7f-77e467c8c882 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:23:16.311472 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-059799b9-e7c9-4494-8e7f-77e467c8c882 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 05:23:16.320205 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application [None req-059799b9-e7c9-4494-8e7f-77e467c8c882 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 25 05:23:16.320205 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:23:16.320205 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:23:16.320205 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:23:16.320205 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:23:16.320205 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:23:16.320205 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:23:16.320205 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:23:16.320205 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:23:16.320205 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:23:16.320205 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:23:16.320205 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:23:16.320205 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 05:23:16.320205 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 05:23:16.320205 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 05:23:16.320205 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 05:23:16.321113 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 05:23:16.321113 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 05:23:16.321113 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 05:23:16.321113 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 05:23:16.321113 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:23:16.321113 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:23:16.321113 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:23:16.321113 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:23:16.321113 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:23:16.321113 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:23:16.321113 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:23:16.321113 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application with Lock( Jul 25 05:23:16.321113 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:23:16.321113 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:23:16.321113 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:23:16.321113 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:23:16.321884 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:23:16.321884 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:23:16.321884 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:23:16.321884 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:23:16.321884 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 05:23:16.321884 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 05:23:16.321884 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 05:23:16.321884 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 05:23:16.321884 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 05:23:16.321884 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 05:23:16.321884 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 25 05:23:16.321884 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Jul 25 05:23:16.321884 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 38/77] 10.222.0.33 () {56 vars in 1114 bytes} [Mon Jul 25 05:23:16 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:23:16.322563 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-059799b9-e7c9-4494-8e7f-77e467c8c882 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:23:16.334436 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:23:16.338164 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5cae436-4a1d-44bd-9f79-28f3abbd4968 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:23:16.372457 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64669f59-076a-40ee-8587-e8b4237339e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:23:16.373082 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64669f59-076a-40ee-8587-e8b4237339e7 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:23:16.373375 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64669f59-076a-40ee-8587-e8b4237339e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:23:16.373659 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64669f59-076a-40ee-8587-e8b4237339e7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:23:16.413120 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:23:16.422047 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-059799b9-e7c9-4494-8e7f-77e467c8c882 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:23:16.441350 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64669f59-076a-40ee-8587-e8b4237339e7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:23:16.442824 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64669f59-076a-40ee-8587-e8b4237339e7 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:23:16.456014 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-509b4bde-77c2-439f-af7e-8bff9ad1b29d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:23:16.456987 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509b4bde-77c2-439f-af7e-8bff9ad1b29d None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:23:16.457328 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509b4bde-77c2-439f-af7e-8bff9ad1b29d None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:23:16.457614 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509b4bde-77c2-439f-af7e-8bff9ad1b29d None admin] PATH_INFO: `/v3/users/1eae540242d94c10b0ad33960a855f08` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:23:16.458839 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-509b4bde-77c2-439f-af7e-8bff9ad1b29d None admin] RBAC: Authorizing `identity:get_user(user_id=1eae540242d94c10b0ad33960a855f08)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:23:16.460391 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-509b4bde-77c2-439f-af7e-8bff9ad1b29d None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:23:16.462078 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 39/78] 10.222.0.33 () {56 vars in 1177 bytes} [Mon Jul 25 05:23:16 2022] GET /identity/v3/users/1eae540242d94c10b0ad33960a855f08 => generated 247 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:23:16.462855 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-509b4bde-77c2-439f-af7e-8bff9ad1b29d None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:23:16.539940 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:23:16.549001 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 40/79] 10.222.0.33 () {58 vars in 1282 bytes} [Mon Jul 25 05:23:16 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 25 05:23:16.550144 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64669f59-076a-40ee-8587-e8b4237339e7 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:23:16.564101 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-509b4bde-77c2-439f-af7e-8bff9ad1b29d None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:23:16.598330 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f18e6ba-9b02-4475-9423-c96d301463d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:23:16.599131 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f18e6ba-9b02-4475-9423-c96d301463d2 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:23:16.599423 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f18e6ba-9b02-4475-9423-c96d301463d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:23:16.599704 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f18e6ba-9b02-4475-9423-c96d301463d2 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:23:16.600956 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f18e6ba-9b02-4475-9423-c96d301463d2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:23:16.602401 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f18e6ba-9b02-4475-9423-c96d301463d2 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:23:16.604238 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 40/80] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Jul 25 05:23:16 2022] GET /identity/v3/domains/default => generated 199 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:23:16.604806 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f18e6ba-9b02-4475-9423-c96d301463d2 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:23:16.633605 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:23:16.653835 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64669f59-076a-40ee-8587-e8b4237339e7 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:23:16.687092 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e2d076d-f3b8-4f72-8fe1-0daeb2c6b7ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:23:16.687889 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2d076d-f3b8-4f72-8fe1-0daeb2c6b7ab None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:23:16.688205 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2d076d-f3b8-4f72-8fe1-0daeb2c6b7ab None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:23:16.688489 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2d076d-f3b8-4f72-8fe1-0daeb2c6b7ab None admin] PATH_INFO: `/v3/roles/member` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:23:16.699038 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e2d076d-f3b8-4f72-8fe1-0daeb2c6b7ab None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 05:23:16.704353 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f18e6ba-9b02-4475-9423-c96d301463d2 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:23:16.707421 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application [None req-1e2d076d-f3b8-4f72-8fe1-0daeb2c6b7ab None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 25 05:23:16.707421 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:23:16.707421 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:23:16.707421 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:23:16.707421 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:23:16.707421 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:23:16.707421 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:23:16.707421 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:23:16.707421 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:23:16.707421 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:23:16.707421 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:23:16.707421 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:23:16.707421 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 05:23:16.707421 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 05:23:16.707421 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 05:23:16.707421 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 05:23:16.708279 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 05:23:16.708279 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 05:23:16.708279 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 05:23:16.708279 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 05:23:16.708279 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:23:16.708279 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:23:16.708279 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:23:16.708279 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:23:16.708279 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:23:16.708279 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:23:16.708279 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:23:16.708279 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application with Lock( Jul 25 05:23:16.708279 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:23:16.708279 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:23:16.708279 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:23:16.708279 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:23:16.709468 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:23:16.709468 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:23:16.709468 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:23:16.709468 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:23:16.709468 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 05:23:16.709468 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 05:23:16.709468 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 05:23:16.709468 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 05:23:16.709468 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 05:23:16.709468 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 05:23:16.709468 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 25 05:23:16.709468 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Jul 25 05:23:16.709468 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 41/81] 10.222.0.33 () {56 vars in 1099 bytes} [Mon Jul 25 05:23:16 2022] GET /identity/v3/roles/member => generated 84 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:23:16.709468 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e2d076d-f3b8-4f72-8fe1-0daeb2c6b7ab None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:23:16.739017 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d45547-733a-493d-9991-a30d09b69b90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:23:16.739693 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d45547-733a-493d-9991-a30d09b69b90 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:23:16.739985 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d45547-733a-493d-9991-a30d09b69b90 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:23:16.740263 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d45547-733a-493d-9991-a30d09b69b90 None admin] PATH_INFO: `/v3/roles` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:23:16.741057 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3d45547-733a-493d-9991-a30d09b69b90 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:23:16.741369 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3d45547-733a-493d-9991-a30d09b69b90 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:23:16.742738 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3d45547-733a-493d-9991-a30d09b69b90 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:23:16.750538 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 41/82] 10.222.0.33 () {56 vars in 1111 bytes} [Mon Jul 25 05:23:16 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:23:16.751653 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d45547-733a-493d-9991-a30d09b69b90 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:23:16.770941 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:23:16.812847 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e2d076d-f3b8-4f72-8fe1-0daeb2c6b7ab None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:23:16.842545 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:23:16.847504 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-585f1786-1dea-4e69-9ff6-7611e0df4675 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:23:16.848189 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585f1786-1dea-4e69-9ff6-7611e0df4675 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:23:16.848475 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585f1786-1dea-4e69-9ff6-7611e0df4675 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:23:16.848763 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585f1786-1dea-4e69-9ff6-7611e0df4675 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:23:16.852941 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d45547-733a-493d-9991-a30d09b69b90 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:23:16.888728 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcfae80a-2901-4ecd-9f02-8afeb42442c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:23:16.889552 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcfae80a-2901-4ecd-9f02-8afeb42442c7 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:23:16.889834 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcfae80a-2901-4ecd-9f02-8afeb42442c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:23:16.890108 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcfae80a-2901-4ecd-9f02-8afeb42442c7 None admin] PATH_INFO: `/v3/projects/4b840bd0540b4512b180fc68fd41cedc` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:23:16.891930 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcfae80a-2901-4ecd-9f02-8afeb42442c7 None admin] RBAC: Authorizing `identity:get_project(project_id=4b840bd0540b4512b180fc68fd41cedc)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:23:16.893544 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcfae80a-2901-4ecd-9f02-8afeb42442c7 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:23:16.895238 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 42/83] 10.222.0.33 () {56 vars in 1186 bytes} [Mon Jul 25 05:23:16 2022] GET /identity/v3/projects/4b840bd0540b4512b180fc68fd41cedc => generated 298 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:23:16.895975 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcfae80a-2901-4ecd-9f02-8afeb42442c7 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:23:16.916542 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-585f1786-1dea-4e69-9ff6-7611e0df4675 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:23:16.918076 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-585f1786-1dea-4e69-9ff6-7611e0df4675 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:23:16.974389 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:23:16.994253 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcfae80a-2901-4ecd-9f02-8afeb42442c7 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:23:17.016651 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 42/84] 10.222.0.33 () {58 vars in 1282 bytes} [Mon Jul 25 05:23:16 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 25 05:23:17.017490 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-585f1786-1dea-4e69-9ff6-7611e0df4675 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:23:17.029562 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89050897-14d9-49cf-a6ec-af7310ca8423 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:23:17.030243 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89050897-14d9-49cf-a6ec-af7310ca8423 None admin] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:23:17.030536 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89050897-14d9-49cf-a6ec-af7310ca8423 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:23:17.030822 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89050897-14d9-49cf-a6ec-af7310ca8423 None admin] PATH_INFO: `/v3/users` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:23:17.031553 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89050897-14d9-49cf-a6ec-af7310ca8423 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:23:17.033322 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89050897-14d9-49cf-a6ec-af7310ca8423 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:23:17.052372 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 43/85] 10.222.0.33 () {60 vars in 1132 bytes} [Mon Jul 25 05:23:16 2022] POST /identity/v3/users => generated 289 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 05:23:17.053242 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89050897-14d9-49cf-a6ec-af7310ca8423 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:23:17.071918 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:23:17.142692 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-585f1786-1dea-4e69-9ff6-7611e0df4675 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:23:17.175608 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89050897-14d9-49cf-a6ec-af7310ca8423 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:23:17.181255 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd0d6bf-d039-4eff-a0ee-3f52c52164cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:23:17.181955 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd0d6bf-d039-4eff-a0ee-3f52c52164cb None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:23:17.182258 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd0d6bf-d039-4eff-a0ee-3f52c52164cb None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:23:17.182543 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd0d6bf-d039-4eff-a0ee-3f52c52164cb None admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:23:17.183325 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfd0d6bf-d039-4eff-a0ee-3f52c52164cb None admin] RBAC: Adding query filter params (name=member) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:23:17.183637 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfd0d6bf-d039-4eff-a0ee-3f52c52164cb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:23:17.185078 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfd0d6bf-d039-4eff-a0ee-3f52c52164cb None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:23:17.197547 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 43/86] 10.222.0.33 () {56 vars in 1101 bytes} [Mon Jul 25 05:23:17 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:23:17.203469 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd0d6bf-d039-4eff-a0ee-3f52c52164cb None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:23:17.214449 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6db02f3-93ab-47f3-b461-1da49e628c50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:23:17.215338 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6db02f3-93ab-47f3-b461-1da49e628c50 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:23:17.215628 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6db02f3-93ab-47f3-b461-1da49e628c50 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:23:17.215909 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6db02f3-93ab-47f3-b461-1da49e628c50 None admin] PATH_INFO: `/v3/users/c25734c1f07644de9355e54cfd5f65cc` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:23:17.217211 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6db02f3-93ab-47f3-b461-1da49e628c50 None admin] RBAC: Authorizing `identity:get_user(user_id=c25734c1f07644de9355e54cfd5f65cc)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:23:17.218680 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6db02f3-93ab-47f3-b461-1da49e628c50 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:23:17.220926 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 44/87] 10.222.0.33 () {56 vars in 1177 bytes} [Mon Jul 25 05:23:17 2022] GET /identity/v3/users/c25734c1f07644de9355e54cfd5f65cc => generated 275 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:23:17.221699 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6db02f3-93ab-47f3-b461-1da49e628c50 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:23:17.237655 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:23:17.323485 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:23:17.338224 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd0d6bf-d039-4eff-a0ee-3f52c52164cb None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:23:17.350407 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6db02f3-93ab-47f3-b461-1da49e628c50 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:23:17.385728 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b9beb70-6ee3-409b-ac1e-da9bf3737774 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:23:17.386436 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9beb70-6ee3-409b-ac1e-da9bf3737774 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:23:17.386725 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9beb70-6ee3-409b-ac1e-da9bf3737774 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:23:17.387002 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9beb70-6ee3-409b-ac1e-da9bf3737774 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:23:17.387850 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b9beb70-6ee3-409b-ac1e-da9bf3737774 None admin] RBAC: Adding query filter params (role.id=5074e74e35f54d4b9bb415c34999a82a, scope.project.id=4b840bd0540b4512b180fc68fd41cedc, user.id=c25734c1f07644de9355e54cfd5f65cc) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:23:17.388166 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b9beb70-6ee3-409b-ac1e-da9bf3737774 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:23:17.389659 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b9beb70-6ee3-409b-ac1e-da9bf3737774 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:23:17.393395 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6728e9dc-76a5-4a3e-8087-3ec7810fc422 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:23:17.394203 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6728e9dc-76a5-4a3e-8087-3ec7810fc422 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:23:17.394932 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6728e9dc-76a5-4a3e-8087-3ec7810fc422 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:23:17.395215 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6728e9dc-76a5-4a3e-8087-3ec7810fc422 None admin] PATH_INFO: `/v3/projects/4b840bd0540b4512b180fc68fd41cedc` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:23:17.396587 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6728e9dc-76a5-4a3e-8087-3ec7810fc422 None admin] RBAC: Authorizing `identity:get_project(project_id=4b840bd0540b4512b180fc68fd41cedc)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:23:17.399944 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6728e9dc-76a5-4a3e-8087-3ec7810fc422 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:23:17.399944 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 45/89] 10.222.0.33 () {56 vars in 1186 bytes} [Mon Jul 25 05:23:17 2022] GET /identity/v3/projects/4b840bd0540b4512b180fc68fd41cedc => generated 298 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:23:17.400160 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 44/89] 10.222.0.33 () {56 vars in 1374 bytes} [Mon Jul 25 05:23:17 2022] GET /identity/v3/role_assignments?user.id=c25734c1f07644de9355e54cfd5f65cc&scope.project.id=4b840bd0540b4512b180fc68fd41cedc&role.id=5074e74e35f54d4b9bb415c34999a82a => generated 259 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:23:17.401001 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b9beb70-6ee3-409b-ac1e-da9bf3737774 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:23:17.405462 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6728e9dc-76a5-4a3e-8087-3ec7810fc422 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:23:17.564323 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6728e9dc-76a5-4a3e-8087-3ec7810fc422 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:23:17.606286 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e543d6e5-4822-423f-b649-e0368a4e2267 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:23:17.607520 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e543d6e5-4822-423f-b649-e0368a4e2267 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:23:17.608017 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e543d6e5-4822-423f-b649-e0368a4e2267 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:23:17.608520 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e543d6e5-4822-423f-b649-e0368a4e2267 None admin] PATH_INFO: `/v3/users/c25734c1f07644de9355e54cfd5f65cc` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:23:17.610272 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e543d6e5-4822-423f-b649-e0368a4e2267 None admin] RBAC: Authorizing `identity:get_user(user_id=c25734c1f07644de9355e54cfd5f65cc)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:23:17.611966 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e543d6e5-4822-423f-b649-e0368a4e2267 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:23:17.613895 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 46/90] 10.222.0.33 () {56 vars in 1177 bytes} [Mon Jul 25 05:23:17 2022] GET /identity/v3/users/c25734c1f07644de9355e54cfd5f65cc => generated 275 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:23:17.614887 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e543d6e5-4822-423f-b649-e0368a4e2267 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:23:17.619708 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:23:17.636381 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:23:17.664933 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b9beb70-6ee3-409b-ac1e-da9bf3737774 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:23:17.733986 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3176fc60-3cb5-4894-b55c-8e835950a0c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:23:17.735262 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3176fc60-3cb5-4894-b55c-8e835950a0c9 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:23:17.737163 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3176fc60-3cb5-4894-b55c-8e835950a0c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:23:17.737680 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3176fc60-3cb5-4894-b55c-8e835950a0c9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:23:17.775273 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e543d6e5-4822-423f-b649-e0368a4e2267 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:23:17.842328 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a2522c-6c19-4870-b595-3c0c123ba4fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:23:17.842991 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a2522c-6c19-4870-b595-3c0c123ba4fa None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:23:17.843279 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a2522c-6c19-4870-b595-3c0c123ba4fa None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:23:17.843559 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a2522c-6c19-4870-b595-3c0c123ba4fa None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:23:17.845181 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5a2522c-6c19-4870-b595-3c0c123ba4fa None admin] RBAC: Adding query filter params (role.id=39e5c666285a42fe8fc2c3afca6260d1, scope.project.id=4b840bd0540b4512b180fc68fd41cedc, user.id=1eae540242d94c10b0ad33960a855f08) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:23:17.845504 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5a2522c-6c19-4870-b595-3c0c123ba4fa None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:23:17.847094 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5a2522c-6c19-4870-b595-3c0c123ba4fa None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:23:17.862121 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 47/91] 10.222.0.33 () {56 vars in 1374 bytes} [Mon Jul 25 05:23:17 2022] GET /identity/v3/role_assignments?user.id=1eae540242d94c10b0ad33960a855f08&scope.project.id=4b840bd0540b4512b180fc68fd41cedc&role.id=39e5c666285a42fe8fc2c3afca6260d1 => generated 259 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:23:17.863423 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a2522c-6c19-4870-b595-3c0c123ba4fa None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:23:17.870476 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3176fc60-3cb5-4894-b55c-8e835950a0c9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:23:17.872554 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3176fc60-3cb5-4894-b55c-8e835950a0c9 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:23:18.049527 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:23:18.070755 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:23:18.093119 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 45/92] 10.222.0.33 () {58 vars in 1282 bytes} [Mon Jul 25 05:23:17 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 693 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 25 05:23:18.100480 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3176fc60-3cb5-4894-b55c-8e835950a0c9 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:23:18.121684 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a2522c-6c19-4870-b595-3c0c123ba4fa None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:23:18.209064 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f5cfeb8-28ab-45f0-8535-88a15eb8bfa5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:23:18.209776 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5cfeb8-28ab-45f0-8535-88a15eb8bfa5 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:23:18.210068 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5cfeb8-28ab-45f0-8535-88a15eb8bfa5 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:23:18.210346 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5cfeb8-28ab-45f0-8535-88a15eb8bfa5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:23:18.250089 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3176fc60-3cb5-4894-b55c-8e835950a0c9 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:23:18.297117 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd5774c9-5b0e-49d1-94d4-a35846b80800 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:23:18.298851 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5774c9-5b0e-49d1-94d4-a35846b80800 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:23:18.299291 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5774c9-5b0e-49d1-94d4-a35846b80800 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:23:18.299795 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5774c9-5b0e-49d1-94d4-a35846b80800 None admin] PATH_INFO: `/v3/projects/4b840bd0540b4512b180fc68fd41cedc` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:23:18.301200 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd5774c9-5b0e-49d1-94d4-a35846b80800 None admin] RBAC: Authorizing `identity:get_project(project_id=4b840bd0540b4512b180fc68fd41cedc)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:23:18.302801 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd5774c9-5b0e-49d1-94d4-a35846b80800 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:23:18.304683 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 46/93] 10.222.0.33 () {56 vars in 1186 bytes} [Mon Jul 25 05:23:18 2022] GET /identity/v3/projects/4b840bd0540b4512b180fc68fd41cedc => generated 298 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:23:18.310446 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd5774c9-5b0e-49d1-94d4-a35846b80800 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:23:18.384803 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f5cfeb8-28ab-45f0-8535-88a15eb8bfa5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:23:18.386881 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f5cfeb8-28ab-45f0-8535-88a15eb8bfa5 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:23:18.411625 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:23:18.507724 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd5774c9-5b0e-49d1-94d4-a35846b80800 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:23:18.548636 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:23:18.571297 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3076d5c-36e0-4817-b58b-788da70e2f61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:23:18.572208 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3076d5c-36e0-4817-b58b-788da70e2f61 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:23:18.572647 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3076d5c-36e0-4817-b58b-788da70e2f61 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:23:18.573095 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3076d5c-36e0-4817-b58b-788da70e2f61 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:23:18.574094 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3076d5c-36e0-4817-b58b-788da70e2f61 None admin] RBAC: Adding query filter params (role.id=02aba1f7cbdf4d6595e18af7fb5727e0, scope.project.id=4b840bd0540b4512b180fc68fd41cedc, user.id=c25734c1f07644de9355e54cfd5f65cc) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:23:18.574574 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3076d5c-36e0-4817-b58b-788da70e2f61 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:23:18.576301 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3076d5c-36e0-4817-b58b-788da70e2f61 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:23:18.582615 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 47/94] 10.222.0.33 () {56 vars in 1374 bytes} [Mon Jul 25 05:23:18 2022] GET /identity/v3/role_assignments?user.id=c25734c1f07644de9355e54cfd5f65cc&scope.project.id=4b840bd0540b4512b180fc68fd41cedc&role.id=02aba1f7cbdf4d6595e18af7fb5727e0 => generated 259 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 05:23:18.582999 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 48/95] 10.222.0.33 () {58 vars in 1282 bytes} [Mon Jul 25 05:23:17 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 720 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 25 05:23:18.588855 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3076d5c-36e0-4817-b58b-788da70e2f61 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:23:18.724968 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3076d5c-36e0-4817-b58b-788da70e2f61 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:23:18.748445 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:23:18.770408 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8355f87-59b7-4dd9-aa7a-9f9b72ae6f4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:23:18.771631 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8355f87-59b7-4dd9-aa7a-9f9b72ae6f4f None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:23:18.772125 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8355f87-59b7-4dd9-aa7a-9f9b72ae6f4f None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:23:18.772625 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8355f87-59b7-4dd9-aa7a-9f9b72ae6f4f None admin] PATH_INFO: `/v3/projects/863d4de6ca1540baa4c4aef54e727bae` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:23:18.774099 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8355f87-59b7-4dd9-aa7a-9f9b72ae6f4f None admin] RBAC: Authorizing `identity:get_project(project_id=863d4de6ca1540baa4c4aef54e727bae)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:23:18.775794 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8355f87-59b7-4dd9-aa7a-9f9b72ae6f4f None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:23:18.778168 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 48/96] 10.222.0.33 () {56 vars in 1186 bytes} [Mon Jul 25 05:23:18 2022] GET /identity/v3/projects/863d4de6ca1540baa4c4aef54e727bae => generated 312 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:23:18.785156 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f5cfeb8-28ab-45f0-8535-88a15eb8bfa5 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:23:18.942706 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f5cfeb8-28ab-45f0-8535-88a15eb8bfa5 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:23:18.989322 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c4887f8-4529-4907-94d0-aef9c77dd7ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:23:18.990337 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4887f8-4529-4907-94d0-aef9c77dd7ea None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:23:18.990849 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4887f8-4529-4907-94d0-aef9c77dd7ea None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:23:18.991335 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4887f8-4529-4907-94d0-aef9c77dd7ea None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:23:18.992404 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c4887f8-4529-4907-94d0-aef9c77dd7ea None admin] RBAC: Adding query filter params (role.id=5074e74e35f54d4b9bb415c34999a82a, scope.project.id=863d4de6ca1540baa4c4aef54e727bae, user.id=c25734c1f07644de9355e54cfd5f65cc) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:23:18.992958 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c4887f8-4529-4907-94d0-aef9c77dd7ea None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:23:18.994610 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c4887f8-4529-4907-94d0-aef9c77dd7ea None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:23:19.003723 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 49/97] 10.222.0.33 () {56 vars in 1374 bytes} [Mon Jul 25 05:23:18 2022] GET /identity/v3/role_assignments?user.id=c25734c1f07644de9355e54cfd5f65cc&scope.project.id=863d4de6ca1540baa4c4aef54e727bae&role.id=5074e74e35f54d4b9bb415c34999a82a => generated 259 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:23:19.819743 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d71297d-55bf-4b12-8911-76fe3c4f3f22 None None] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:23:19.820624 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d71297d-55bf-4b12-8911-76fe3c4f3f22 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:23:19.821134 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d71297d-55bf-4b12-8911-76fe3c4f3f22 None None] PATH_INFO: `/` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:23:19.823024 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 49/98] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jul 25 05:23:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:23:19.829211 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca43909-0e9b-4e05-bc50-76a9a027bc94 None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:23:19.830040 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca43909-0e9b-4e05-bc50-76a9a027bc94 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:23:19.830525 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca43909-0e9b-4e05-bc50-76a9a027bc94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:23:19.852582 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-eca43909-0e9b-4e05-bc50-76a9a027bc94 None None] MFA Rules not processed for user `1eae540242d94c10b0ad33960a855f08`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:23:19.854121 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-eca43909-0e9b-4e05-bc50-76a9a027bc94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:23:19.902852 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:23:19.934330 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 50/99] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jul 25 05:23:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 05:23:19.970428 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d71297d-55bf-4b12-8911-76fe3c4f3f22 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:23:20.099097 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d71297d-55bf-4b12-8911-76fe3c4f3f22 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:23:20.134769 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b9d78a-da4b-4f16-be38-aa75fe2868e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:23:20.135917 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b9d78a-da4b-4f16-be38-aa75fe2868e9 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:23:20.136434 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b9d78a-da4b-4f16-be38-aa75fe2868e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:23:20.136949 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b9d78a-da4b-4f16-be38-aa75fe2868e9 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:23:20.138390 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01b9d78a-da4b-4f16-be38-aa75fe2868e9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:23:20.140045 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01b9d78a-da4b-4f16-be38-aa75fe2868e9 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:23:20.142258 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 50/100] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Jul 25 05:23:19 2022] GET /identity/v3/domains/default => generated 199 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:23:20.150154 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca43909-0e9b-4e05-bc50-76a9a027bc94 None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:23:20.238847 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4116c438-094c-4bd2-8ae6-c2bbeb26b080 None None] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:23:20.239500 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4116c438-094c-4bd2-8ae6-c2bbeb26b080 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:23:20.239988 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4116c438-094c-4bd2-8ae6-c2bbeb26b080 None None] PATH_INFO: `/` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:23:20.241614 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 51/101] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jul 25 05:23:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 25 05:23:20.258152 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ba48d0-211f-4904-809e-a4e96a80bca6 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:23:20.258776 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ba48d0-211f-4904-809e-a4e96a80bca6 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:23:20.259257 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ba48d0-211f-4904-809e-a4e96a80bca6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:23:20.280583 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-a7ba48d0-211f-4904-809e-a4e96a80bca6 None None] MFA Rules not processed for user `1eae540242d94c10b0ad33960a855f08`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:23:20.284550 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-a7ba48d0-211f-4904-809e-a4e96a80bca6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:23:20.299103 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca43909-0e9b-4e05-bc50-76a9a027bc94 None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:23:20.372706 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:23:20.374980 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a6c5d4-4457-4510-8f6d-c9c7f0fbbed7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:23:20.375826 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a6c5d4-4457-4510-8f6d-c9c7f0fbbed7 None admin] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:23:20.376140 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a6c5d4-4457-4510-8f6d-c9c7f0fbbed7 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:23:20.376382 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a6c5d4-4457-4510-8f6d-c9c7f0fbbed7 None admin] PATH_INFO: `/v3/projects` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:23:20.377161 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88a6c5d4-4457-4510-8f6d-c9c7f0fbbed7 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:23:20.378684 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88a6c5d4-4457-4510-8f6d-c9c7f0fbbed7 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:23:20.414495 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 52/102] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jul 25 05:23:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 25 05:23:20.415872 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66cdb316-7243-445d-83fa-ca01e2508a4d None None] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:23:20.416172 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66cdb316-7243-445d-83fa-ca01e2508a4d None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:23:20.416460 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66cdb316-7243-445d-83fa-ca01e2508a4d None None] PATH_INFO: `/` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:23:20.417798 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 53/103] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jul 25 05:23:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:23:20.425816 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58062a93-6051-46a1-8fa5-893dfcef680a None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:23:20.426110 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58062a93-6051-46a1-8fa5-893dfcef680a None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:23:20.426399 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58062a93-6051-46a1-8fa5-893dfcef680a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:23:20.429039 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 51/104] 10.222.0.33 () {60 vars in 1141 bytes} [Mon Jul 25 05:23:20 2022] POST /identity/v3/projects => generated 302 bytes in 279 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 05:23:20.450470 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-58062a93-6051-46a1-8fa5-893dfcef680a None None] MFA Rules not processed for user `1eae540242d94c10b0ad33960a855f08`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:23:20.451784 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-58062a93-6051-46a1-8fa5-893dfcef680a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:23:20.466528 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a6c5d4-4457-4510-8f6d-c9c7f0fbbed7 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:23:20.533256 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:23:20.591464 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 54/105] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jul 25 05:23:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 05:23:20.641435 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a6c5d4-4457-4510-8f6d-c9c7f0fbbed7 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:23:20.662298 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58062a93-6051-46a1-8fa5-893dfcef680a None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:23:20.727925 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-103e9c71-e845-40d9-8408-3b92c3b331d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:23:20.728747 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103e9c71-e845-40d9-8408-3b92c3b331d5 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:23:20.729126 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103e9c71-e845-40d9-8408-3b92c3b331d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:23:20.729416 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103e9c71-e845-40d9-8408-3b92c3b331d5 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:23:20.750899 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-103e9c71-e845-40d9-8408-3b92c3b331d5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 05:23:20.763850 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application [None req-103e9c71-e845-40d9-8408-3b92c3b331d5 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 25 05:23:20.763850 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:23:20.763850 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:23:20.763850 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:23:20.763850 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:23:20.763850 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:23:20.763850 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:23:20.763850 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:23:20.763850 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:23:20.763850 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:23:20.763850 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:23:20.763850 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:23:20.763850 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 05:23:20.763850 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 05:23:20.763850 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 05:23:20.763850 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 05:23:20.764742 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 05:23:20.764742 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 05:23:20.764742 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 05:23:20.764742 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 05:23:20.764742 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:23:20.764742 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:23:20.764742 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:23:20.764742 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:23:20.764742 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:23:20.764742 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:23:20.764742 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:23:20.764742 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application with Lock( Jul 25 05:23:20.764742 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:23:20.764742 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:23:20.764742 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:23:20.764742 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:23:20.765551 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:23:20.765551 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:23:20.765551 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:23:20.765551 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:23:20.765551 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 05:23:20.765551 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 05:23:20.765551 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 05:23:20.765551 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 05:23:20.765551 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 05:23:20.765551 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 05:23:20.765551 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 25 05:23:20.765551 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Jul 25 05:23:20.765551 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 52/106] 10.222.0.33 () {56 vars in 1099 bytes} [Mon Jul 25 05:23:20 2022] GET /identity/v3/roles/reader => generated 84 bytes in 299 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 25 05:23:20.772710 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-103e9c71-e845-40d9-8408-3b92c3b331d5 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:23:20.850710 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58062a93-6051-46a1-8fa5-893dfcef680a None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:23:20.867542 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:23:20.882466 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:23:20.898099 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-103e9c71-e845-40d9-8408-3b92c3b331d5 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:23:20.900008 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13cc0de2-bb2a-4aa6-b1f8-caeedcb04489 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:23:20.902004 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13cc0de2-bb2a-4aa6-b1f8-caeedcb04489 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:23:20.902441 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13cc0de2-bb2a-4aa6-b1f8-caeedcb04489 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:23:20.902865 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13cc0de2-bb2a-4aa6-b1f8-caeedcb04489 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:23:20.918067 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13cc0de2-bb2a-4aa6-b1f8-caeedcb04489 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 05:23:20.929697 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application [None req-13cc0de2-bb2a-4aa6-b1f8-caeedcb04489 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 25 05:23:20.929697 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:23:20.929697 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:23:20.929697 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:23:20.929697 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:23:20.929697 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:23:20.929697 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:23:20.929697 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:23:20.929697 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:23:20.929697 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:23:20.929697 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:23:20.929697 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:23:20.929697 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 05:23:20.929697 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 05:23:20.929697 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 05:23:20.929697 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 05:23:20.930782 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 05:23:20.930782 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 05:23:20.930782 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 05:23:20.930782 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 05:23:20.930782 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:23:20.930782 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:23:20.930782 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:23:20.930782 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:23:20.930782 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:23:20.930782 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:23:20.930782 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:23:20.930782 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application with Lock( Jul 25 05:23:20.930782 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:23:20.930782 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:23:20.930782 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:23:20.930782 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:23:20.931711 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:23:20.931711 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:23:20.931711 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:23:20.931711 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:23:20.931711 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 05:23:20.931711 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 05:23:20.931711 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 05:23:20.931711 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 05:23:20.931711 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 05:23:20.931711 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 05:23:20.931711 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 25 05:23:20.931711 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Jul 25 05:23:20.934579 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 55/107] 10.222.0.33 () {56 vars in 1099 bytes} [Mon Jul 25 05:23:20 2022] GET /identity/v3/roles/member => generated 84 bytes in 273 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 25 05:23:20.936127 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6b32f3-ac95-4cc9-ade7-93f0991556c5 None None] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:23:20.936579 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6b32f3-ac95-4cc9-ade7-93f0991556c5 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:23:20.937013 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6b32f3-ac95-4cc9-ade7-93f0991556c5 None None] PATH_INFO: `/` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:23:20.939249 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 56/108] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jul 25 05:23:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:23:20.944224 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d2c10e1-dc63-4c77-951b-781a23080f0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:23:20.945045 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2c10e1-dc63-4c77-951b-781a23080f0d None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:23:20.945445 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2c10e1-dc63-4c77-951b-781a23080f0d None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:23:20.945855 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2c10e1-dc63-4c77-951b-781a23080f0d None admin] PATH_INFO: `/v3/roles` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:23:20.946687 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d2c10e1-dc63-4c77-951b-781a23080f0d None admin] RBAC: Adding query filter params (name=reader) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:23:20.947095 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d2c10e1-dc63-4c77-951b-781a23080f0d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:23:20.948554 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d2c10e1-dc63-4c77-951b-781a23080f0d None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:23:20.953563 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ea5008-ec99-49ec-8409-ac36e11def39 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:23:20.954233 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ea5008-ec99-49ec-8409-ac36e11def39 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:23:20.955247 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ea5008-ec99-49ec-8409-ac36e11def39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:23:20.961999 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 53/109] 10.222.0.33 () {56 vars in 1101 bytes} [Mon Jul 25 05:23:20 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:23:20.962911 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d2c10e1-dc63-4c77-951b-781a23080f0d None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:23:20.976569 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-f1ea5008-ec99-49ec-8409-ac36e11def39 None None] MFA Rules not processed for user `1eae540242d94c10b0ad33960a855f08`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:23:20.978605 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-f1ea5008-ec99-49ec-8409-ac36e11def39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:23:21.067365 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 57/110] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jul 25 05:23:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 25 05:23:21.067365 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ea5008-ec99-49ec-8409-ac36e11def39 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:23:21.142877 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d2c10e1-dc63-4c77-951b-781a23080f0d None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:23:21.178734 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7811a286-6ea6-4b7b-a2cc-86bc7e629ed9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:23:21.179339 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7811a286-6ea6-4b7b-a2cc-86bc7e629ed9 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:23:21.179633 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7811a286-6ea6-4b7b-a2cc-86bc7e629ed9 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:23:21.179916 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7811a286-6ea6-4b7b-a2cc-86bc7e629ed9 None admin] PATH_INFO: `/v3/roles` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:23:21.180679 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7811a286-6ea6-4b7b-a2cc-86bc7e629ed9 None admin] RBAC: Adding query filter params (name=member) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:23:21.181024 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7811a286-6ea6-4b7b-a2cc-86bc7e629ed9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:23:21.182420 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7811a286-6ea6-4b7b-a2cc-86bc7e629ed9 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:23:21.191940 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 54/111] 10.222.0.33 () {56 vars in 1101 bytes} [Mon Jul 25 05:23:20 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:23:21.192537 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7811a286-6ea6-4b7b-a2cc-86bc7e629ed9 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:23:21.207445 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:23:21.240523 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ea5008-ec99-49ec-8409-ac36e11def39 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:23:21.244172 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:23:21.282562 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d06469b-0c27-4474-9192-2c9a9a72610c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:23:21.283447 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d06469b-0c27-4474-9192-2c9a9a72610c None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:23:21.283736 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d06469b-0c27-4474-9192-2c9a9a72610c None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:23:21.284026 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d06469b-0c27-4474-9192-2c9a9a72610c None admin] PATH_INFO: `/v3/users/6fd6f234e1a54a4480852ab2c80c37c3` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:23:21.301473 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d06469b-0c27-4474-9192-2c9a9a72610c None admin] RBAC: Authorizing `identity:get_user(user_id=6fd6f234e1a54a4480852ab2c80c37c3)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:23:21.303109 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d06469b-0c27-4474-9192-2c9a9a72610c None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:23:21.305224 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 58/112] 10.222.0.33 () {56 vars in 1177 bytes} [Mon Jul 25 05:23:21 2022] GET /identity/v3/users/6fd6f234e1a54a4480852ab2c80c37c3 => generated 289 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:23:21.306009 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d06469b-0c27-4474-9192-2c9a9a72610c None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:23:21.363638 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7811a286-6ea6-4b7b-a2cc-86bc7e629ed9 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:23:21.436665 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d06469b-0c27-4474-9192-2c9a9a72610c None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:23:21.461046 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-647224df-06a4-4d42-bf91-d7f4562e566f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:23:21.461863 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647224df-06a4-4d42-bf91-d7f4562e566f None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:23:21.462161 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647224df-06a4-4d42-bf91-d7f4562e566f None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:23:21.462449 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647224df-06a4-4d42-bf91-d7f4562e566f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:23:21.489985 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-647224df-06a4-4d42-bf91-d7f4562e566f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 05:23:21.491701 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c414594-52e4-40a9-a234-6d65f9b06eb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:23:21.492597 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c414594-52e4-40a9-a234-6d65f9b06eb7 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:23:21.492919 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c414594-52e4-40a9-a234-6d65f9b06eb7 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:23:21.493204 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c414594-52e4-40a9-a234-6d65f9b06eb7 None admin] PATH_INFO: `/v3/users/c25734c1f07644de9355e54cfd5f65cc` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:23:21.494433 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c414594-52e4-40a9-a234-6d65f9b06eb7 None admin] RBAC: Authorizing `identity:get_user(user_id=c25734c1f07644de9355e54cfd5f65cc)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:23:21.496019 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c414594-52e4-40a9-a234-6d65f9b06eb7 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:23:21.497878 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 59/113] 10.222.0.33 () {56 vars in 1177 bytes} [Mon Jul 25 05:23:21 2022] GET /identity/v3/users/c25734c1f07644de9355e54cfd5f65cc => generated 275 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:23:21.498068 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application [None req-647224df-06a4-4d42-bf91-d7f4562e566f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 05:23:21.498068 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:23:21.498068 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:23:21.498068 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:23:21.498068 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:23:21.498068 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:23:21.498068 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:23:21.498068 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:23:21.498068 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:23:21.498068 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:23:21.498068 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:23:21.498068 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:23:21.498068 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 05:23:21.498068 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 05:23:21.498068 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 05:23:21.498068 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 05:23:21.498897 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 05:23:21.498897 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 05:23:21.498897 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 05:23:21.498897 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 05:23:21.498897 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:23:21.498897 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:23:21.498897 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:23:21.498897 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:23:21.498897 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:23:21.498897 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:23:21.498897 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:23:21.498897 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application with Lock( Jul 25 05:23:21.498897 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:23:21.498897 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:23:21.498897 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:23:21.498897 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:23:21.499660 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:23:21.499660 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:23:21.499660 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:23:21.499660 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:23:21.499660 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 05:23:21.499660 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 05:23:21.499660 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 05:23:21.499660 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 05:23:21.499660 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 05:23:21.499660 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 05:23:21.499660 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 05:23:21.499660 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Jul 25 05:23:21.500845 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c414594-52e4-40a9-a234-6d65f9b06eb7 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:23:21.500917 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 55/114] 10.222.0.33 () {56 vars in 1096 bytes} [Mon Jul 25 05:23:21 2022] GET /identity/v3/roles/admin => generated 83 bytes in 307 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:23:21.501507 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1135c88c-3545-428b-9f94-e914c5161ac5 None None] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:23:21.505079 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1135c88c-3545-428b-9f94-e914c5161ac5 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:23:21.505079 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1135c88c-3545-428b-9f94-e914c5161ac5 None None] PATH_INFO: `/` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:23:21.505079 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 56/115] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jul 25 05:23:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:23:21.507507 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1135c88c-3545-428b-9f94-e914c5161ac5 None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:23:21.538384 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:23:21.589245 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:23:21.630893 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c414594-52e4-40a9-a234-6d65f9b06eb7 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:23:21.635818 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1135c88c-3545-428b-9f94-e914c5161ac5 None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:23:21.668491 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c484e9-1d63-43c0-b71c-d64b50a41df3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:23:21.669200 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c484e9-1d63-43c0-b71c-d64b50a41df3 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:23:21.669490 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c484e9-1d63-43c0-b71c-d64b50a41df3 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:23:21.669773 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c484e9-1d63-43c0-b71c-d64b50a41df3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:23:21.682306 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-205a3326-e840-49ba-95a6-faa8be253589 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:23:21.683466 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205a3326-e840-49ba-95a6-faa8be253589 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:23:21.683808 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205a3326-e840-49ba-95a6-faa8be253589 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:23:21.684413 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205a3326-e840-49ba-95a6-faa8be253589 None admin] PATH_INFO: `/v3/roles` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:23:21.685214 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-205a3326-e840-49ba-95a6-faa8be253589 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:23:21.685823 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-205a3326-e840-49ba-95a6-faa8be253589 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:23:21.688680 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-205a3326-e840-49ba-95a6-faa8be253589 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:23:21.701668 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 57/116] 10.222.0.33 () {56 vars in 1099 bytes} [Mon Jul 25 05:23:21 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:23:21.702465 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-205a3326-e840-49ba-95a6-faa8be253589 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:23:21.762917 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c484e9-1d63-43c0-b71c-d64b50a41df3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:23:21.765184 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c484e9-1d63-43c0-b71c-d64b50a41df3 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:23:21.803561 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:23:21.823412 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-205a3326-e840-49ba-95a6-faa8be253589 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:23:21.860977 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f3a620c-11bb-4c1c-9da8-c84c283f155b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:23:21.862184 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3a620c-11bb-4c1c-9da8-c84c283f155b None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:23:21.862700 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3a620c-11bb-4c1c-9da8-c84c283f155b None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:23:21.863176 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3a620c-11bb-4c1c-9da8-c84c283f155b None admin] PATH_INFO: `/v3/projects/4b840bd0540b4512b180fc68fd41cedc` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:23:21.864605 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f3a620c-11bb-4c1c-9da8-c84c283f155b None admin] RBAC: Authorizing `identity:get_project(project_id=4b840bd0540b4512b180fc68fd41cedc)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:23:21.866275 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f3a620c-11bb-4c1c-9da8-c84c283f155b None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:23:21.868295 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 58/117] 10.222.0.33 () {56 vars in 1186 bytes} [Mon Jul 25 05:23:21 2022] GET /identity/v3/projects/4b840bd0540b4512b180fc68fd41cedc => generated 298 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:23:21.869856 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59cdd315-6051-41b3-a368-cf4c43726005 None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:23:21.870350 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59cdd315-6051-41b3-a368-cf4c43726005 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:23:21.870881 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59cdd315-6051-41b3-a368-cf4c43726005 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:23:21.876021 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:23:21.885972 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 60/118] 10.222.0.33 () {58 vars in 1282 bytes} [Mon Jul 25 05:23:21 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 388 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 25 05:23:21.886826 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c484e9-1d63-43c0-b71c-d64b50a41df3 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:23:21.891540 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-59cdd315-6051-41b3-a368-cf4c43726005 None None] MFA Rules not processed for user `1eae540242d94c10b0ad33960a855f08`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:23:21.893369 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-59cdd315-6051-41b3-a368-cf4c43726005 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:23:21.964941 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 59/119] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jul 25 05:23:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 05:23:21.974030 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59cdd315-6051-41b3-a368-cf4c43726005 None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:23:21.992933 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c484e9-1d63-43c0-b71c-d64b50a41df3 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:23:22.021425 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:23:22.029976 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e8fc6ab-ed03-4729-b0a4-1eda85d95733 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:23:22.030871 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8fc6ab-ed03-4729-b0a4-1eda85d95733 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:23:22.031192 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8fc6ab-ed03-4729-b0a4-1eda85d95733 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:23:22.031548 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8fc6ab-ed03-4729-b0a4-1eda85d95733 None admin] PATH_INFO: `/v3/users/1eae540242d94c10b0ad33960a855f08` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:23:22.033134 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e8fc6ab-ed03-4729-b0a4-1eda85d95733 None admin] RBAC: Authorizing `identity:get_user(user_id=1eae540242d94c10b0ad33960a855f08)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:23:22.035320 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e8fc6ab-ed03-4729-b0a4-1eda85d95733 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:23:22.038264 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 61/120] 10.222.0.33 () {56 vars in 1177 bytes} [Mon Jul 25 05:23:21 2022] GET /identity/v3/users/1eae540242d94c10b0ad33960a855f08 => generated 247 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:23:22.038264 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e8fc6ab-ed03-4729-b0a4-1eda85d95733 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:23:22.106708 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59cdd315-6051-41b3-a368-cf4c43726005 None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:23:22.144337 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:23:22.151731 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f55f572-92a5-4262-99e4-84eb4547826d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:23:22.152886 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f55f572-92a5-4262-99e4-84eb4547826d None admin] REQUEST_METHOD: `PUT` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:23:22.153046 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e8fc6ab-ed03-4729-b0a4-1eda85d95733 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:23:22.153501 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f55f572-92a5-4262-99e4-84eb4547826d None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:23:22.153951 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f55f572-92a5-4262-99e4-84eb4547826d None admin] PATH_INFO: `/v3/projects/4b840bd0540b4512b180fc68fd41cedc/users/c25734c1f07644de9355e54cfd5f65cc/roles/5074e74e35f54d4b9bb415c34999a82a` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:23:22.157204 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f55f572-92a5-4262-99e4-84eb4547826d None admin] RBAC: Authorizing `identity:create_grant(project_id=4b840bd0540b4512b180fc68fd41cedc, user_id=c25734c1f07644de9355e54cfd5f65cc, role_id=5074e74e35f54d4b9bb415c34999a82a)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:23:22.158872 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f55f572-92a5-4262-99e4-84eb4547826d None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:23:22.180895 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 60/121] 10.222.0.33 () {58 vars in 1439 bytes} [Mon Jul 25 05:23:21 2022] PUT /identity/v3/projects/4b840bd0540b4512b180fc68fd41cedc/users/c25734c1f07644de9355e54cfd5f65cc/roles/5074e74e35f54d4b9bb415c34999a82a => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 05:23:22.181946 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f55f572-92a5-4262-99e4-84eb4547826d None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:23:22.195379 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf11f62-3662-47ca-89ef-7b1d51aa3132 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:23:22.196262 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf11f62-3662-47ca-89ef-7b1d51aa3132 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:23:22.196550 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf11f62-3662-47ca-89ef-7b1d51aa3132 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:23:22.196885 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf11f62-3662-47ca-89ef-7b1d51aa3132 None admin] PATH_INFO: `/v3/projects/4b840bd0540b4512b180fc68fd41cedc` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:23:22.198069 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaf11f62-3662-47ca-89ef-7b1d51aa3132 None admin] RBAC: Authorizing `identity:get_project(project_id=4b840bd0540b4512b180fc68fd41cedc)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:23:22.199537 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaf11f62-3662-47ca-89ef-7b1d51aa3132 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:23:22.201867 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 62/122] 10.222.0.33 () {56 vars in 1186 bytes} [Mon Jul 25 05:23:22 2022] GET /identity/v3/projects/4b840bd0540b4512b180fc68fd41cedc => generated 298 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:23:22.202694 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf11f62-3662-47ca-89ef-7b1d51aa3132 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:23:22.288296 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:23:22.339360 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f55f572-92a5-4262-99e4-84eb4547826d None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:23:22.340228 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf11f62-3662-47ca-89ef-7b1d51aa3132 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:23:22.382179 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3467702-34f3-410b-88cb-f400f9e82e06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:23:22.384812 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3467702-34f3-410b-88cb-f400f9e82e06 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:23:22.384812 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3467702-34f3-410b-88cb-f400f9e82e06 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:23:22.384812 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3467702-34f3-410b-88cb-f400f9e82e06 None admin] PATH_INFO: `/v3/projects/4b840bd0540b4512b180fc68fd41cedc` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:23:22.385128 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec19eb74-48c7-4192-80f3-a3324e92a046 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:23:22.385128 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec19eb74-48c7-4192-80f3-a3324e92a046 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:23:22.385128 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec19eb74-48c7-4192-80f3-a3324e92a046 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:23:22.385322 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3467702-34f3-410b-88cb-f400f9e82e06 None admin] RBAC: Authorizing `identity:get_project(project_id=4b840bd0540b4512b180fc68fd41cedc)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:23:22.385389 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec19eb74-48c7-4192-80f3-a3324e92a046 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:23:22.386290 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3467702-34f3-410b-88cb-f400f9e82e06 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:23:22.387824 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 63/123] 10.222.0.33 () {56 vars in 1186 bytes} [Mon Jul 25 05:23:22 2022] GET /identity/v3/projects/4b840bd0540b4512b180fc68fd41cedc => generated 298 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:23:22.389023 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e07396-d084-490f-a799-768e33e10541 None None] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:23:22.389318 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e07396-d084-490f-a799-768e33e10541 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:23:22.389604 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e07396-d084-490f-a799-768e33e10541 None None] PATH_INFO: `/` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:23:22.391024 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 64/124] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jul 25 05:23:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:23:22.391947 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e07396-d084-490f-a799-768e33e10541 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:23:22.404891 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec19eb74-48c7-4192-80f3-a3324e92a046 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 05:23:22.415912 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application [None req-ec19eb74-48c7-4192-80f3-a3324e92a046 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 25 05:23:22.415912 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:23:22.415912 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:23:22.415912 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:23:22.415912 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:23:22.415912 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:23:22.415912 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:23:22.415912 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:23:22.415912 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:23:22.415912 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:23:22.415912 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:23:22.415912 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:23:22.415912 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 05:23:22.415912 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 05:23:22.415912 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 05:23:22.415912 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 05:23:22.416883 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 05:23:22.416883 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 05:23:22.416883 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 05:23:22.416883 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 05:23:22.416883 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:23:22.416883 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:23:22.416883 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:23:22.416883 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:23:22.416883 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:23:22.416883 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:23:22.416883 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:23:22.416883 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application with Lock( Jul 25 05:23:22.416883 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:23:22.416883 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:23:22.416883 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:23:22.416883 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:23:22.417655 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:23:22.417655 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:23:22.417655 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:23:22.417655 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:23:22.417655 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 05:23:22.417655 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 05:23:22.417655 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 05:23:22.417655 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 05:23:22.417655 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 05:23:22.417655 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 05:23:22.417655 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 25 05:23:22.417655 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Jul 25 05:23:22.417655 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 61/125] 10.222.0.33 () {56 vars in 1114 bytes} [Mon Jul 25 05:23:22 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:23:22.419087 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec19eb74-48c7-4192-80f3-a3324e92a046 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:23:22.484983 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:23:22.509039 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e07396-d084-490f-a799-768e33e10541 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:23:22.539461 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec19eb74-48c7-4192-80f3-a3324e92a046 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:23:22.540806 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:23:22.546210 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a870f2f3-714e-4750-ab65-b396a4997437 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:23:22.547674 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a870f2f3-714e-4750-ab65-b396a4997437 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:23:22.547875 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a870f2f3-714e-4750-ab65-b396a4997437 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:23:22.548355 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a870f2f3-714e-4750-ab65-b396a4997437 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:23:22.549641 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a870f2f3-714e-4750-ab65-b396a4997437 None admin] RBAC: Adding query filter params (role.id=be79956018b14baba7a021e2adf27084, scope.project.id=4b840bd0540b4512b180fc68fd41cedc, user.id=6fd6f234e1a54a4480852ab2c80c37c3) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:23:22.550334 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a870f2f3-714e-4750-ab65-b396a4997437 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:23:22.552206 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a870f2f3-714e-4750-ab65-b396a4997437 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:23:22.558477 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 65/126] 10.222.0.33 () {56 vars in 1374 bytes} [Mon Jul 25 05:23:22 2022] GET /identity/v3/role_assignments?user.id=6fd6f234e1a54a4480852ab2c80c37c3&scope.project.id=4b840bd0540b4512b180fc68fd41cedc&role.id=be79956018b14baba7a021e2adf27084 => generated 259 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:23:22.559838 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa80df8-d55f-4644-b908-dd2e32067442 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:23:22.565594 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa80df8-d55f-4644-b908-dd2e32067442 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:23:22.566009 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa80df8-d55f-4644-b908-dd2e32067442 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:23:22.590137 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-8fa80df8-d55f-4644-b908-dd2e32067442 None None] MFA Rules not processed for user `1eae540242d94c10b0ad33960a855f08`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:23:22.590638 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a697e5ac-067c-4ce9-adc4-8f89bcf3dbdd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:23:22.591727 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a697e5ac-067c-4ce9-adc4-8f89bcf3dbdd None admin] REQUEST_METHOD: `PUT` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:23:22.591841 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-8fa80df8-d55f-4644-b908-dd2e32067442 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:23:22.592285 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a697e5ac-067c-4ce9-adc4-8f89bcf3dbdd None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:23:22.592713 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a697e5ac-067c-4ce9-adc4-8f89bcf3dbdd None admin] PATH_INFO: `/v3/projects/4b840bd0540b4512b180fc68fd41cedc/users/1eae540242d94c10b0ad33960a855f08/roles/39e5c666285a42fe8fc2c3afca6260d1` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:23:22.595803 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a697e5ac-067c-4ce9-adc4-8f89bcf3dbdd None admin] RBAC: Authorizing `identity:create_grant(project_id=4b840bd0540b4512b180fc68fd41cedc, user_id=1eae540242d94c10b0ad33960a855f08, role_id=39e5c666285a42fe8fc2c3afca6260d1)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:23:22.597499 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a697e5ac-067c-4ce9-adc4-8f89bcf3dbdd None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:23:22.632386 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 62/127] 10.222.0.33 () {58 vars in 1439 bytes} [Mon Jul 25 05:23:22 2022] PUT /identity/v3/projects/4b840bd0540b4512b180fc68fd41cedc/users/1eae540242d94c10b0ad33960a855f08/roles/39e5c666285a42fe8fc2c3afca6260d1 => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 05:23:22.641020 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a697e5ac-067c-4ce9-adc4-8f89bcf3dbdd None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:23:22.672303 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 66/128] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jul 25 05:23:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 05:23:22.696148 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa80df8-d55f-4644-b908-dd2e32067442 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:23:22.808556 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa80df8-d55f-4644-b908-dd2e32067442 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:23:22.826393 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a697e5ac-067c-4ce9-adc4-8f89bcf3dbdd None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:23:22.846967 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:23:22.870505 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee2a88a-c44c-4619-8f13-9ad16ee48b06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:23:22.871426 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee2a88a-c44c-4619-8f13-9ad16ee48b06 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:23:22.871726 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee2a88a-c44c-4619-8f13-9ad16ee48b06 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:23:22.872016 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee2a88a-c44c-4619-8f13-9ad16ee48b06 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:23:22.873247 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:23:22.897890 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ee2a88a-c44c-4619-8f13-9ad16ee48b06 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 05:23:22.904490 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15bead89-cafa-4880-84a3-5ace46ea17c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:23:22.906920 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15bead89-cafa-4880-84a3-5ace46ea17c6 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:23:22.907249 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15bead89-cafa-4880-84a3-5ace46ea17c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:23:22.907598 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15bead89-cafa-4880-84a3-5ace46ea17c6 None admin] PATH_INFO: `/v3/roles` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:23:22.908384 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15bead89-cafa-4880-84a3-5ace46ea17c6 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:23:22.908706 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15bead89-cafa-4880-84a3-5ace46ea17c6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:23:22.910153 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15bead89-cafa-4880-84a3-5ace46ea17c6 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:23:22.910768 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application [None req-0ee2a88a-c44c-4619-8f13-9ad16ee48b06 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 25 05:23:22.910768 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:23:22.910768 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:23:22.910768 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:23:22.910768 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:23:22.910768 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:23:22.910768 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:23:22.910768 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:23:22.910768 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:23:22.910768 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:23:22.910768 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:23:22.910768 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:23:22.910768 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 05:23:22.910768 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 05:23:22.910768 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 05:23:22.910768 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 05:23:22.911603 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 05:23:22.911603 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 05:23:22.911603 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 05:23:22.911603 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 05:23:22.911603 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:23:22.911603 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:23:22.911603 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:23:22.911603 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:23:22.911603 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:23:22.911603 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:23:22.911603 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:23:22.911603 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application with Lock( Jul 25 05:23:22.911603 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:23:22.911603 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:23:22.911603 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:23:22.911603 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:23:22.912378 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:23:22.912378 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:23:22.912378 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:23:22.912378 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:23:22.912378 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 05:23:22.912378 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 05:23:22.912378 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 05:23:22.912378 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 05:23:22.912378 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 05:23:22.912378 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 05:23:22.912378 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 25 05:23:22.912378 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Jul 25 05:23:22.912378 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 67/129] 10.222.0.33 () {56 vars in 1099 bytes} [Mon Jul 25 05:23:22 2022] GET /identity/v3/roles/member => generated 84 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:23:22.918081 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee2a88a-c44c-4619-8f13-9ad16ee48b06 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:23:22.925651 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 63/130] 10.222.0.33 () {56 vars in 1111 bytes} [Mon Jul 25 05:23:22 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 292 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:23:22.932320 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15bead89-cafa-4880-84a3-5ace46ea17c6 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:23:23.083574 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee2a88a-c44c-4619-8f13-9ad16ee48b06 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:23:23.124379 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15bead89-cafa-4880-84a3-5ace46ea17c6 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:23:23.152670 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fdbf2bb-7737-4dca-8fd3-5901a3fc143a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:23:23.153742 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fdbf2bb-7737-4dca-8fd3-5901a3fc143a None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:23:23.154226 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fdbf2bb-7737-4dca-8fd3-5901a3fc143a None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:23:23.154699 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fdbf2bb-7737-4dca-8fd3-5901a3fc143a None admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:23:23.155652 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fdbf2bb-7737-4dca-8fd3-5901a3fc143a None admin] RBAC: Adding query filter params (name=member) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:23:23.156827 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fdbf2bb-7737-4dca-8fd3-5901a3fc143a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:23:23.159498 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fdbf2bb-7737-4dca-8fd3-5901a3fc143a None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:23:23.177171 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 68/131] 10.222.0.33 () {56 vars in 1101 bytes} [Mon Jul 25 05:23:22 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:23:23.184815 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fdbf2bb-7737-4dca-8fd3-5901a3fc143a None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:23:23.198354 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc47b52-6d4a-4bb0-bd23-96dfc7314dd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:23:23.199669 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc47b52-6d4a-4bb0-bd23-96dfc7314dd2 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:23:23.200152 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc47b52-6d4a-4bb0-bd23-96dfc7314dd2 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:23:23.200647 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc47b52-6d4a-4bb0-bd23-96dfc7314dd2 None admin] PATH_INFO: `/v3/users/c25734c1f07644de9355e54cfd5f65cc` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:23:23.204521 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bc47b52-6d4a-4bb0-bd23-96dfc7314dd2 None admin] RBAC: Authorizing `identity:get_user(user_id=c25734c1f07644de9355e54cfd5f65cc)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:23:23.206316 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bc47b52-6d4a-4bb0-bd23-96dfc7314dd2 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:23:23.209642 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 64/132] 10.222.0.33 () {56 vars in 1177 bytes} [Mon Jul 25 05:23:22 2022] GET /identity/v3/users/c25734c1f07644de9355e54cfd5f65cc => generated 275 bytes in 278 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 05:23:23.215029 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc47b52-6d4a-4bb0-bd23-96dfc7314dd2 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:23:23.257754 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:23:23.279127 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:23:23.316809 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fdbf2bb-7737-4dca-8fd3-5901a3fc143a None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:23:23.353893 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd5f4f7a-be6a-471f-baa3-3561ec4d3dd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:23:23.355098 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5f4f7a-be6a-471f-baa3-3561ec4d3dd4 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:23:23.355579 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5f4f7a-be6a-471f-baa3-3561ec4d3dd4 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:23:23.356048 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5f4f7a-be6a-471f-baa3-3561ec4d3dd4 None admin] PATH_INFO: `/v3/users/c25734c1f07644de9355e54cfd5f65cc` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:23:23.357567 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd5f4f7a-be6a-471f-baa3-3561ec4d3dd4 None admin] RBAC: Authorizing `identity:get_user(user_id=c25734c1f07644de9355e54cfd5f65cc)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:23:23.357671 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc47b52-6d4a-4bb0-bd23-96dfc7314dd2 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:23:23.359475 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd5f4f7a-be6a-471f-baa3-3561ec4d3dd4 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:23:23.361394 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 69/133] 10.222.0.33 () {56 vars in 1177 bytes} [Mon Jul 25 05:23:23 2022] GET /identity/v3/users/c25734c1f07644de9355e54cfd5f65cc => generated 275 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 05:23:23.362811 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5165ca-bff2-4ed1-abf1-f9a702e811a2 None None] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:23:23.363318 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5165ca-bff2-4ed1-abf1-f9a702e811a2 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:23:23.363788 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5165ca-bff2-4ed1-abf1-f9a702e811a2 None None] PATH_INFO: `/` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:23:23.365324 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 70/134] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jul 25 05:23:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:23:23.367199 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c5165ca-bff2-4ed1-abf1-f9a702e811a2 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:23:23.405921 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c4685e-02c2-4d5a-b898-f152a04dcd04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:23:23.407053 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c4685e-02c2-4d5a-b898-f152a04dcd04 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:23:23.407355 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c4685e-02c2-4d5a-b898-f152a04dcd04 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:23:23.407645 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c4685e-02c2-4d5a-b898-f152a04dcd04 None admin] PATH_INFO: `/v3/projects/4b840bd0540b4512b180fc68fd41cedc` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:23:23.409037 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0c4685e-02c2-4d5a-b898-f152a04dcd04 None admin] RBAC: Authorizing `identity:get_project(project_id=4b840bd0540b4512b180fc68fd41cedc)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:23:23.411377 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0c4685e-02c2-4d5a-b898-f152a04dcd04 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:23:23.414644 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 65/135] 10.222.0.33 () {56 vars in 1186 bytes} [Mon Jul 25 05:23:23 2022] GET /identity/v3/projects/4b840bd0540b4512b180fc68fd41cedc => generated 298 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 05:23:23.416654 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f72bfee-1d07-432a-9a25-7c774558e7d9 None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:23:23.417212 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f72bfee-1d07-432a-9a25-7c774558e7d9 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:23:23.418155 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f72bfee-1d07-432a-9a25-7c774558e7d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:23:23.441611 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-1f72bfee-1d07-432a-9a25-7c774558e7d9 None None] MFA Rules not processed for user `1eae540242d94c10b0ad33960a855f08`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:23:23.442823 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-1f72bfee-1d07-432a-9a25-7c774558e7d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:23:23.499476 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c5165ca-bff2-4ed1-abf1-f9a702e811a2 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:23:23.525176 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 66/136] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jul 25 05:23:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 05:23:23.526066 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:23:23.529279 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f72bfee-1d07-432a-9a25-7c774558e7d9 None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:23:23.563362 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d6d487-6ec6-411a-b229-27d4c024f2b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:23:23.564249 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d6d487-6ec6-411a-b229-27d4c024f2b4 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:23:23.564524 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d6d487-6ec6-411a-b229-27d4c024f2b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:23:23.564821 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d6d487-6ec6-411a-b229-27d4c024f2b4 None admin] PATH_INFO: `/v3/projects/863d4de6ca1540baa4c4aef54e727bae` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:23:23.566337 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5d6d487-6ec6-411a-b229-27d4c024f2b4 None admin] RBAC: Authorizing `identity:get_project(project_id=863d4de6ca1540baa4c4aef54e727bae)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:23:23.567859 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5d6d487-6ec6-411a-b229-27d4c024f2b4 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:23:23.569923 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 71/137] 10.222.0.33 () {56 vars in 1186 bytes} [Mon Jul 25 05:23:23 2022] GET /identity/v3/projects/863d4de6ca1540baa4c4aef54e727bae => generated 312 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:23:23.576440 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d6d487-6ec6-411a-b229-27d4c024f2b4 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:23:23.599898 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:23:23.666715 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f72bfee-1d07-432a-9a25-7c774558e7d9 None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:23:23.704828 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8840d60-4c3f-451f-ac7b-02490a0977c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:23:23.706178 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8840d60-4c3f-451f-ac7b-02490a0977c5 None admin] REQUEST_METHOD: `PUT` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:23:23.706685 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8840d60-4c3f-451f-ac7b-02490a0977c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:23:23.707165 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8840d60-4c3f-451f-ac7b-02490a0977c5 None admin] PATH_INFO: `/v3/projects/4b840bd0540b4512b180fc68fd41cedc/users/c25734c1f07644de9355e54cfd5f65cc/roles/02aba1f7cbdf4d6595e18af7fb5727e0` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:23:23.709773 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8840d60-4c3f-451f-ac7b-02490a0977c5 None admin] RBAC: Authorizing `identity:create_grant(project_id=4b840bd0540b4512b180fc68fd41cedc, user_id=c25734c1f07644de9355e54cfd5f65cc, role_id=02aba1f7cbdf4d6595e18af7fb5727e0)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:23:23.711544 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8840d60-4c3f-451f-ac7b-02490a0977c5 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:23:23.725457 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 67/138] 10.222.0.33 () {58 vars in 1439 bytes} [Mon Jul 25 05:23:23 2022] PUT /identity/v3/projects/4b840bd0540b4512b180fc68fd41cedc/users/c25734c1f07644de9355e54cfd5f65cc/roles/02aba1f7cbdf4d6595e18af7fb5727e0 => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 05:23:23.726622 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8840d60-4c3f-451f-ac7b-02490a0977c5 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:23:23.847875 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:23:23.852167 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8840d60-4c3f-451f-ac7b-02490a0977c5 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:23:23.863206 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d6d487-6ec6-411a-b229-27d4c024f2b4 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:23:23.898560 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b415f0-1a07-4746-926f-30304dccac1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:23:23.900004 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b415f0-1a07-4746-926f-30304dccac1a None admin] REQUEST_METHOD: `PUT` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:23:23.900522 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b415f0-1a07-4746-926f-30304dccac1a None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:23:23.901020 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b415f0-1a07-4746-926f-30304dccac1a None admin] PATH_INFO: `/v3/projects/863d4de6ca1540baa4c4aef54e727bae/users/c25734c1f07644de9355e54cfd5f65cc/roles/5074e74e35f54d4b9bb415c34999a82a` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:23:23.903903 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68b415f0-1a07-4746-926f-30304dccac1a None admin] RBAC: Authorizing `identity:create_grant(project_id=863d4de6ca1540baa4c4aef54e727bae, user_id=c25734c1f07644de9355e54cfd5f65cc, role_id=5074e74e35f54d4b9bb415c34999a82a)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:23:23.905964 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68b415f0-1a07-4746-926f-30304dccac1a None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:23:23.918102 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e07641a-6835-41db-b548-f6fa4691552a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:23:23.918607 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e07641a-6835-41db-b548-f6fa4691552a None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:23:23.918830 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e07641a-6835-41db-b548-f6fa4691552a None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:23:23.919073 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e07641a-6835-41db-b548-f6fa4691552a None admin] PATH_INFO: `/v3/domains/default` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:23:23.920298 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e07641a-6835-41db-b548-f6fa4691552a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:23:23.922412 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e07641a-6835-41db-b548-f6fa4691552a None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:23:23.924538 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 72/139] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Jul 25 05:23:23 2022] GET /identity/v3/domains/default => generated 199 bytes in 348 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:23:23.929947 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e07641a-6835-41db-b548-f6fa4691552a None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:23:23.941778 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 68/140] 10.222.0.33 () {58 vars in 1439 bytes} [Mon Jul 25 05:23:23 2022] PUT /identity/v3/projects/863d4de6ca1540baa4c4aef54e727bae/users/c25734c1f07644de9355e54cfd5f65cc/roles/5074e74e35f54d4b9bb415c34999a82a => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 05:23:24.016166 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:23:24.075577 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e07641a-6835-41db-b548-f6fa4691552a None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:23:24.120509 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e2b2ecf-8938-4499-bbf8-6a5ce1d44bc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:23:24.121280 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2b2ecf-8938-4499-bbf8-6a5ce1d44bc4 None admin] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:23:24.121583 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2b2ecf-8938-4499-bbf8-6a5ce1d44bc4 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:23:24.121873 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2b2ecf-8938-4499-bbf8-6a5ce1d44bc4 None admin] PATH_INFO: `/v3/users` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:23:24.122625 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e2b2ecf-8938-4499-bbf8-6a5ce1d44bc4 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:23:24.124114 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e2b2ecf-8938-4499-bbf8-6a5ce1d44bc4 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:23:24.157664 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 73/141] 10.222.0.33 () {60 vars in 1132 bytes} [Mon Jul 25 05:23:23 2022] POST /identity/v3/users => generated 283 bytes in 228 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 05:23:24.917042 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d75717b-45de-4bb6-805e-d39b7e9984b6 None None] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:23:24.917576 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d75717b-45de-4bb6-805e-d39b7e9984b6 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:23:24.917652 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d75717b-45de-4bb6-805e-d39b7e9984b6 None None] PATH_INFO: `/` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:23:24.918979 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 69/142] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jul 25 05:23:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:23:24.925575 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d4bd46-96de-4584-9e8d-9fc76f6f1a03 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:23:24.925996 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d4bd46-96de-4584-9e8d-9fc76f6f1a03 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:23:24.926201 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d4bd46-96de-4584-9e8d-9fc76f6f1a03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:23:24.947393 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-37d4bd46-96de-4584-9e8d-9fc76f6f1a03 None None] MFA Rules not processed for user `1eae540242d94c10b0ad33960a855f08`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:23:24.948643 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-37d4bd46-96de-4584-9e8d-9fc76f6f1a03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:23:25.037490 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 74/143] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jul 25 05:23:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 05:23:25.067007 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d75717b-45de-4bb6-805e-d39b7e9984b6 None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:23:25.204323 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d75717b-45de-4bb6-805e-d39b7e9984b6 None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:23:25.246147 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c520e79e-1cef-4935-b8d4-14ecffb8d5d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:23:25.246950 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c520e79e-1cef-4935-b8d4-14ecffb8d5d8 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:23:25.247234 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c520e79e-1cef-4935-b8d4-14ecffb8d5d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:23:25.247509 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c520e79e-1cef-4935-b8d4-14ecffb8d5d8 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:23:25.259192 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c520e79e-1cef-4935-b8d4-14ecffb8d5d8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 05:23:25.268034 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application [None req-c520e79e-1cef-4935-b8d4-14ecffb8d5d8 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 25 05:23:25.268034 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:23:25.268034 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:23:25.268034 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:23:25.268034 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:23:25.268034 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:23:25.268034 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:23:25.268034 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:23:25.268034 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:23:25.268034 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:23:25.268034 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:23:25.268034 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:23:25.268034 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 05:23:25.268034 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 05:23:25.268034 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 05:23:25.268034 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 05:23:25.269171 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 05:23:25.269171 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 05:23:25.269171 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 05:23:25.269171 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 05:23:25.269171 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:23:25.269171 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:23:25.269171 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:23:25.269171 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:23:25.269171 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:23:25.269171 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:23:25.269171 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:23:25.269171 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application with Lock( Jul 25 05:23:25.269171 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:23:25.269171 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:23:25.269171 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:23:25.269171 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:23:25.270084 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:23:25.270084 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:23:25.270084 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:23:25.270084 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:23:25.270084 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 05:23:25.270084 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 05:23:25.270084 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 05:23:25.270084 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 05:23:25.270084 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 05:23:25.270084 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 05:23:25.270084 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 25 05:23:25.270084 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Jul 25 05:23:25.270084 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 70/144] 10.222.0.33 () {56 vars in 1099 bytes} [Mon Jul 25 05:23:25 2022] GET /identity/v3/roles/member => generated 84 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:23:25.276712 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d4bd46-96de-4584-9e8d-9fc76f6f1a03 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:23:25.351847 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:23:25.423716 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d4bd46-96de-4584-9e8d-9fc76f6f1a03 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:23:25.464553 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e92a02f-9b91-44b0-ad75-38851bffd0a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:23:25.465279 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e92a02f-9b91-44b0-ad75-38851bffd0a3 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:23:25.465573 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e92a02f-9b91-44b0-ad75-38851bffd0a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:23:25.465861 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e92a02f-9b91-44b0-ad75-38851bffd0a3 None admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:23:25.466639 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e92a02f-9b91-44b0-ad75-38851bffd0a3 None admin] RBAC: Adding query filter params (name=member) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:23:25.466967 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e92a02f-9b91-44b0-ad75-38851bffd0a3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:23:25.468405 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e92a02f-9b91-44b0-ad75-38851bffd0a3 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:23:25.477767 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 75/145] 10.222.0.33 () {56 vars in 1101 bytes} [Mon Jul 25 05:23:25 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:23:25.483416 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c520e79e-1cef-4935-b8d4-14ecffb8d5d8 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:23:25.520181 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:23:25.628300 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c520e79e-1cef-4935-b8d4-14ecffb8d5d8 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:23:25.666033 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ec0099-101e-47fe-86f9-18a583cbf260 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:23:25.666915 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ec0099-101e-47fe-86f9-18a583cbf260 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:23:25.667201 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ec0099-101e-47fe-86f9-18a583cbf260 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:23:25.667473 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ec0099-101e-47fe-86f9-18a583cbf260 None admin] PATH_INFO: `/v3/users/c25734c1f07644de9355e54cfd5f65cc` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:23:25.668681 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24ec0099-101e-47fe-86f9-18a583cbf260 None admin] RBAC: Authorizing `identity:get_user(user_id=c25734c1f07644de9355e54cfd5f65cc)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:23:25.670166 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24ec0099-101e-47fe-86f9-18a583cbf260 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:23:25.672418 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 71/146] 10.222.0.33 () {56 vars in 1177 bytes} [Mon Jul 25 05:23:25 2022] GET /identity/v3/users/c25734c1f07644de9355e54cfd5f65cc => generated 275 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:23:25.681902 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e92a02f-9b91-44b0-ad75-38851bffd0a3 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:23:25.841262 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:23:25.845706 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e92a02f-9b91-44b0-ad75-38851bffd0a3 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:23:25.886881 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ead23b-0e6b-4b2e-a6a9-98b1fa1a4ba1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:23:25.887463 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ead23b-0e6b-4b2e-a6a9-98b1fa1a4ba1 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:23:25.887746 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ead23b-0e6b-4b2e-a6a9-98b1fa1a4ba1 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:23:25.888023 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ead23b-0e6b-4b2e-a6a9-98b1fa1a4ba1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:23:25.975825 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22ead23b-0e6b-4b2e-a6a9-98b1fa1a4ba1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:23:25.977324 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22ead23b-0e6b-4b2e-a6a9-98b1fa1a4ba1 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:23:26.013784 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844a7734-5281-4b5c-8fb0-becb1632c809 None None] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:23:26.014324 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844a7734-5281-4b5c-8fb0-becb1632c809 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:23:26.014324 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844a7734-5281-4b5c-8fb0-becb1632c809 None None] PATH_INFO: `/` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:23:26.015939 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 72/147] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jul 25 05:23:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:23:26.025271 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70638b63-efa3-4abe-addc-b16679adc01a None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:23:26.025562 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70638b63-efa3-4abe-addc-b16679adc01a None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:23:26.025963 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70638b63-efa3-4abe-addc-b16679adc01a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:23:26.055306 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-70638b63-efa3-4abe-addc-b16679adc01a None None] MFA Rules not processed for user `1eae540242d94c10b0ad33960a855f08`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:23:26.056511 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-70638b63-efa3-4abe-addc-b16679adc01a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:23:26.149901 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:23:26.157476 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 76/148] 10.222.0.33 () {58 vars in 1282 bytes} [Mon Jul 25 05:23:25 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 476 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 25 05:23:26.162865 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ead23b-0e6b-4b2e-a6a9-98b1fa1a4ba1 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:23:26.165041 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 73/149] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jul 25 05:23:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 25 05:23:26.189255 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70638b63-efa3-4abe-addc-b16679adc01a None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:23:26.209251 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:23:26.298053 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70638b63-efa3-4abe-addc-b16679adc01a None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:23:26.335906 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a32d09-45d6-4dcf-9a85-9570584ea814 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:23:26.336766 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a32d09-45d6-4dcf-9a85-9570584ea814 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:23:26.337077 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a32d09-45d6-4dcf-9a85-9570584ea814 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:23:26.337351 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a32d09-45d6-4dcf-9a85-9570584ea814 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:23:26.350964 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23a32d09-45d6-4dcf-9a85-9570584ea814 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 05:23:26.359716 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application [None req-23a32d09-45d6-4dcf-9a85-9570584ea814 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 05:23:26.359716 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:23:26.359716 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:23:26.359716 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:23:26.359716 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:23:26.359716 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:23:26.359716 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:23:26.359716 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:23:26.359716 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:23:26.359716 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:23:26.359716 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:23:26.359716 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:23:26.359716 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 05:23:26.359716 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 05:23:26.359716 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 05:23:26.359716 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 05:23:26.361822 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 05:23:26.361822 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 05:23:26.361822 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 05:23:26.361822 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 05:23:26.361822 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:23:26.361822 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:23:26.361822 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:23:26.361822 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:23:26.361822 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:23:26.361822 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:23:26.361822 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:23:26.361822 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application with Lock( Jul 25 05:23:26.361822 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:23:26.361822 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:23:26.361822 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:23:26.361822 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:23:26.363790 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:23:26.363790 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:23:26.363790 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:23:26.363790 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:23:26.363790 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 05:23:26.363790 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 05:23:26.363790 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 05:23:26.363790 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 05:23:26.363790 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 05:23:26.363790 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 05:23:26.363790 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 05:23:26.363790 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Jul 25 05:23:26.365400 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ead23b-0e6b-4b2e-a6a9-98b1fa1a4ba1 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:23:26.365556 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 74/150] 10.222.0.33 () {56 vars in 1096 bytes} [Mon Jul 25 05:23:26 2022] GET /identity/v3/roles/admin => generated 83 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:23:26.368808 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce7c7d5-447a-43c8-8c4f-a46b07ad9949 None None] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:23:26.368808 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce7c7d5-447a-43c8-8c4f-a46b07ad9949 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:23:26.368808 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce7c7d5-447a-43c8-8c4f-a46b07ad9949 None None] PATH_INFO: `/` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:23:26.368808 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 75/151] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jul 25 05:23:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:23:26.371596 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce7c7d5-447a-43c8-8c4f-a46b07ad9949 None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:23:26.473159 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:23:26.481508 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40700ab9-735b-4973-b669-12265a194dfd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:23:26.482487 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40700ab9-735b-4973-b669-12265a194dfd None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:23:26.482776 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40700ab9-735b-4973-b669-12265a194dfd None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:23:26.483053 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40700ab9-735b-4973-b669-12265a194dfd None admin] PATH_INFO: `/v3/projects/4b840bd0540b4512b180fc68fd41cedc` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:23:26.489598 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40700ab9-735b-4973-b669-12265a194dfd None admin] RBAC: Authorizing `identity:get_project(project_id=4b840bd0540b4512b180fc68fd41cedc)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:23:26.491070 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40700ab9-735b-4973-b669-12265a194dfd None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:23:26.493024 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 77/152] 10.222.0.33 () {56 vars in 1186 bytes} [Mon Jul 25 05:23:26 2022] GET /identity/v3/projects/4b840bd0540b4512b180fc68fd41cedc => generated 298 bytes in 330 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 05:23:26.494039 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696d278d-7222-48e0-ac30-0aecbe8411df None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:23:26.494323 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696d278d-7222-48e0-ac30-0aecbe8411df None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:23:26.494602 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696d278d-7222-48e0-ac30-0aecbe8411df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:23:26.514952 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce7c7d5-447a-43c8-8c4f-a46b07ad9949 None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:23:26.524973 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-696d278d-7222-48e0-ac30-0aecbe8411df None None] MFA Rules not processed for user `1eae540242d94c10b0ad33960a855f08`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:23:26.526308 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-696d278d-7222-48e0-ac30-0aecbe8411df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:23:26.558134 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a864e49-7b02-4541-b834-dea6aebf24f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:23:26.558822 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a864e49-7b02-4541-b834-dea6aebf24f9 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:23:26.559107 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a864e49-7b02-4541-b834-dea6aebf24f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:23:26.559384 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a864e49-7b02-4541-b834-dea6aebf24f9 None admin] PATH_INFO: `/v3/roles` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:23:26.560249 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a864e49-7b02-4541-b834-dea6aebf24f9 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:23:26.560561 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a864e49-7b02-4541-b834-dea6aebf24f9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:23:26.561974 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a864e49-7b02-4541-b834-dea6aebf24f9 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:23:26.578277 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 76/153] 10.222.0.33 () {56 vars in 1099 bytes} [Mon Jul 25 05:23:26 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:23:26.579193 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a864e49-7b02-4541-b834-dea6aebf24f9 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:23:26.603955 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:23:26.618754 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 78/154] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jul 25 05:23:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 05:23:26.619844 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-696d278d-7222-48e0-ac30-0aecbe8411df None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:23:26.735808 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a864e49-7b02-4541-b834-dea6aebf24f9 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:23:26.761425 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-696d278d-7222-48e0-ac30-0aecbe8411df None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:23:26.795866 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ab59ec-2a6c-4b45-8bad-6ec4aafa2dcc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:23:26.797186 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ab59ec-2a6c-4b45-8bad-6ec4aafa2dcc None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:23:26.797961 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ab59ec-2a6c-4b45-8bad-6ec4aafa2dcc None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:23:26.798365 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ab59ec-2a6c-4b45-8bad-6ec4aafa2dcc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:23:26.799370 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8ab59ec-2a6c-4b45-8bad-6ec4aafa2dcc None admin] RBAC: Adding query filter params (role.id=5074e74e35f54d4b9bb415c34999a82a, scope.project.id=4b840bd0540b4512b180fc68fd41cedc, user.id=c25734c1f07644de9355e54cfd5f65cc) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:23:26.799824 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8ab59ec-2a6c-4b45-8bad-6ec4aafa2dcc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:23:26.801458 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8ab59ec-2a6c-4b45-8bad-6ec4aafa2dcc None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:23:26.813502 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 77/155] 10.222.0.33 () {56 vars in 1374 bytes} [Mon Jul 25 05:23:26 2022] GET /identity/v3/role_assignments?user.id=c25734c1f07644de9355e54cfd5f65cc&scope.project.id=4b840bd0540b4512b180fc68fd41cedc&role.id=5074e74e35f54d4b9bb415c34999a82a => generated 608 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:23:26.816169 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ab59ec-2a6c-4b45-8bad-6ec4aafa2dcc None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:23:26.820813 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a98f026-1569-4e65-9828-30053132038b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:23:26.820813 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a98f026-1569-4e65-9828-30053132038b None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:23:26.820813 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a98f026-1569-4e65-9828-30053132038b None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:23:26.820813 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a98f026-1569-4e65-9828-30053132038b None admin] PATH_INFO: `/v3/users/1eae540242d94c10b0ad33960a855f08` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:23:26.824752 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:23:26.824928 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a98f026-1569-4e65-9828-30053132038b None admin] RBAC: Authorizing `identity:get_user(user_id=1eae540242d94c10b0ad33960a855f08)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:23:26.826677 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a98f026-1569-4e65-9828-30053132038b None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:23:26.828649 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 79/156] 10.222.0.33 () {56 vars in 1177 bytes} [Mon Jul 25 05:23:26 2022] GET /identity/v3/users/1eae540242d94c10b0ad33960a855f08 => generated 247 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:23:26.848715 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a98f026-1569-4e65-9828-30053132038b None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:23:26.970947 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ab59ec-2a6c-4b45-8bad-6ec4aafa2dcc None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:23:26.977883 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:23:27.009725 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31cb13ce-8e2f-4197-b2c8-6262c5a04c3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:23:27.010788 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31cb13ce-8e2f-4197-b2c8-6262c5a04c3d None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:23:27.011212 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31cb13ce-8e2f-4197-b2c8-6262c5a04c3d None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:23:27.011625 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31cb13ce-8e2f-4197-b2c8-6262c5a04c3d None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:23:27.024296 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31cb13ce-8e2f-4197-b2c8-6262c5a04c3d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 05:23:27.033690 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application [None req-31cb13ce-8e2f-4197-b2c8-6262c5a04c3d None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 25 05:23:27.033690 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:23:27.033690 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:23:27.033690 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:23:27.033690 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:23:27.033690 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:23:27.033690 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:23:27.033690 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:23:27.033690 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:23:27.033690 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:23:27.033690 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:23:27.033690 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:23:27.033690 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 05:23:27.033690 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 05:23:27.033690 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 05:23:27.033690 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 05:23:27.034975 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 05:23:27.034975 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 05:23:27.034975 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 05:23:27.034975 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 05:23:27.034975 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:23:27.034975 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:23:27.034975 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:23:27.034975 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:23:27.034975 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:23:27.034975 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:23:27.034975 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:23:27.034975 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application with Lock( Jul 25 05:23:27.034975 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:23:27.034975 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:23:27.034975 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:23:27.034975 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:23:27.036003 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:23:27.036003 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:23:27.036003 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:23:27.036003 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:23:27.036003 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 05:23:27.036003 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 05:23:27.036003 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 05:23:27.036003 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 05:23:27.036003 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 05:23:27.036003 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 05:23:27.036003 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 25 05:23:27.036003 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Jul 25 05:23:27.038187 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 78/157] 10.222.0.33 () {56 vars in 1099 bytes} [Mon Jul 25 05:23:26 2022] GET /identity/v3/roles/reader => generated 84 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:23:27.044075 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31cb13ce-8e2f-4197-b2c8-6262c5a04c3d None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:23:27.073002 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a98f026-1569-4e65-9828-30053132038b None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:23:27.177546 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b5c8683-312b-4e1a-9de3-568156a93caf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:23:27.178228 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5c8683-312b-4e1a-9de3-568156a93caf None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:23:27.178512 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5c8683-312b-4e1a-9de3-568156a93caf None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:23:27.178790 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5c8683-312b-4e1a-9de3-568156a93caf None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:23:27.230253 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:23:27.329347 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31cb13ce-8e2f-4197-b2c8-6262c5a04c3d None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:23:27.349870 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b5c8683-312b-4e1a-9de3-568156a93caf None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:23:27.351360 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b5c8683-312b-4e1a-9de3-568156a93caf None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:23:27.425547 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c059b1b-69cc-4844-a15e-b7c9777e8af4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:23:27.426245 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c059b1b-69cc-4844-a15e-b7c9777e8af4 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:23:27.426538 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c059b1b-69cc-4844-a15e-b7c9777e8af4 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:23:27.426823 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c059b1b-69cc-4844-a15e-b7c9777e8af4 None admin] PATH_INFO: `/v3/roles` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:23:27.427597 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c059b1b-69cc-4844-a15e-b7c9777e8af4 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:23:27.427912 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c059b1b-69cc-4844-a15e-b7c9777e8af4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:23:27.429365 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c059b1b-69cc-4844-a15e-b7c9777e8af4 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:23:27.448184 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 79/158] 10.222.0.33 () {56 vars in 1101 bytes} [Mon Jul 25 05:23:27 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 404 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:23:27.449354 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec19d03-38ca-41b1-95ed-e42e608d3da5 None None] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:23:27.449649 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec19d03-38ca-41b1-95ed-e42e608d3da5 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:23:27.449952 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec19d03-38ca-41b1-95ed-e42e608d3da5 None None] PATH_INFO: `/` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:23:27.451450 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 80/159] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jul 25 05:23:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:23:27.456470 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec19d03-38ca-41b1-95ed-e42e608d3da5 None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:23:27.497263 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:23:27.596159 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 80/160] 10.222.0.33 () {58 vars in 1282 bytes} [Mon Jul 25 05:23:26 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 748 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 25 05:23:27.597510 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19dd1be8-f96d-4889-9bb7-0b9cf85db35f None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:23:27.597813 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19dd1be8-f96d-4889-9bb7-0b9cf85db35f None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:23:27.598098 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19dd1be8-f96d-4889-9bb7-0b9cf85db35f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:23:27.626292 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec19d03-38ca-41b1-95ed-e42e608d3da5 None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:23:27.629395 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-19dd1be8-f96d-4889-9bb7-0b9cf85db35f None None] MFA Rules not processed for user `1eae540242d94c10b0ad33960a855f08`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:23:27.630609 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-19dd1be8-f96d-4889-9bb7-0b9cf85db35f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:23:27.661143 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:23:27.670112 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dea96f1-4f7a-4b40-910f-07beee24c2ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:23:27.671003 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dea96f1-4f7a-4b40-910f-07beee24c2ea None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:23:27.671338 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dea96f1-4f7a-4b40-910f-07beee24c2ea None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:23:27.671811 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dea96f1-4f7a-4b40-910f-07beee24c2ea None admin] PATH_INFO: `/v3/users/6fd6f234e1a54a4480852ab2c80c37c3` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:23:27.673250 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dea96f1-4f7a-4b40-910f-07beee24c2ea None admin] RBAC: Authorizing `identity:get_user(user_id=6fd6f234e1a54a4480852ab2c80c37c3)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:23:27.675208 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dea96f1-4f7a-4b40-910f-07beee24c2ea None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:23:27.677263 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 81/161] 10.222.0.33 () {56 vars in 1177 bytes} [Mon Jul 25 05:23:27 2022] GET /identity/v3/users/6fd6f234e1a54a4480852ab2c80c37c3 => generated 289 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 05:23:27.678625 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dea96f1-4f7a-4b40-910f-07beee24c2ea None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:23:27.791947 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dea96f1-4f7a-4b40-910f-07beee24c2ea None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:23:27.829690 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e039b6e-bb58-4d21-b85a-7f52c964d5d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:23:27.831096 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e039b6e-bb58-4d21-b85a-7f52c964d5d8 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:23:27.831443 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e039b6e-bb58-4d21-b85a-7f52c964d5d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:23:27.831786 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e039b6e-bb58-4d21-b85a-7f52c964d5d8 None admin] PATH_INFO: `/v3/projects/4b840bd0540b4512b180fc68fd41cedc` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:23:27.833079 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e039b6e-bb58-4d21-b85a-7f52c964d5d8 None admin] RBAC: Authorizing `identity:get_project(project_id=4b840bd0540b4512b180fc68fd41cedc)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:23:27.835350 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e039b6e-bb58-4d21-b85a-7f52c964d5d8 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:23:27.837329 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 82/162] 10.222.0.33 () {56 vars in 1186 bytes} [Mon Jul 25 05:23:27 2022] GET /identity/v3/projects/4b840bd0540b4512b180fc68fd41cedc => generated 298 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:23:27.838735 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8904f5c3-0b9b-4517-9409-f91124c75192 None None] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:23:27.839087 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8904f5c3-0b9b-4517-9409-f91124c75192 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:23:27.839431 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8904f5c3-0b9b-4517-9409-f91124c75192 None None] PATH_INFO: `/` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:23:27.840899 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 83/163] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jul 25 05:23:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:23:27.841727 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8904f5c3-0b9b-4517-9409-f91124c75192 None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:23:27.846316 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 81/164] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jul 25 05:23:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 250 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 05:23:27.847057 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19dd1be8-f96d-4889-9bb7-0b9cf85db35f None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:23:27.913219 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:23:27.974767 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8904f5c3-0b9b-4517-9409-f91124c75192 None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:23:28.045519 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:23:28.050235 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe30ae8-80da-439d-accd-98f32ad968f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:23:28.051343 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe30ae8-80da-439d-accd-98f32ad968f2 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:23:28.051718 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe30ae8-80da-439d-accd-98f32ad968f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:23:28.052089 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe30ae8-80da-439d-accd-98f32ad968f2 None admin] PATH_INFO: `/v3/projects/4b840bd0540b4512b180fc68fd41cedc` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:23:28.054686 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19dd1be8-f96d-4889-9bb7-0b9cf85db35f None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:23:28.055505 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abe30ae8-80da-439d-accd-98f32ad968f2 None admin] RBAC: Authorizing `identity:get_project(project_id=4b840bd0540b4512b180fc68fd41cedc)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:23:28.056984 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abe30ae8-80da-439d-accd-98f32ad968f2 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:23:28.058864 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 84/165] 10.222.0.33 () {56 vars in 1186 bytes} [Mon Jul 25 05:23:27 2022] GET /identity/v3/projects/4b840bd0540b4512b180fc68fd41cedc => generated 298 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:23:28.064814 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624a8815-73cc-4385-a35e-9d12c4f49143 None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:23:28.065367 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624a8815-73cc-4385-a35e-9d12c4f49143 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:23:28.065648 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624a8815-73cc-4385-a35e-9d12c4f49143 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:23:28.094125 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d84cfb1-33b6-4848-94d8-1903928eb118 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:23:28.095169 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d84cfb1-33b6-4848-94d8-1903928eb118 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:23:28.095660 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d84cfb1-33b6-4848-94d8-1903928eb118 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:23:28.096236 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d84cfb1-33b6-4848-94d8-1903928eb118 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:23:28.097407 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d84cfb1-33b6-4848-94d8-1903928eb118 None admin] RBAC: Adding query filter params (role.id=39e5c666285a42fe8fc2c3afca6260d1, scope.project.id=4b840bd0540b4512b180fc68fd41cedc, user.id=1eae540242d94c10b0ad33960a855f08) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:23:28.098015 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d84cfb1-33b6-4848-94d8-1903928eb118 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:23:28.099712 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d84cfb1-33b6-4848-94d8-1903928eb118 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:23:28.101996 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-624a8815-73cc-4385-a35e-9d12c4f49143 None None] MFA Rules not processed for user `1eae540242d94c10b0ad33960a855f08`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:23:28.103110 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-624a8815-73cc-4385-a35e-9d12c4f49143 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:23:28.107419 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 82/166] 10.222.0.33 () {56 vars in 1374 bytes} [Mon Jul 25 05:23:27 2022] GET /identity/v3/role_assignments?user.id=1eae540242d94c10b0ad33960a855f08&scope.project.id=4b840bd0540b4512b180fc68fd41cedc&role.id=39e5c666285a42fe8fc2c3afca6260d1 => generated 608 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:23:28.108473 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d84cfb1-33b6-4848-94d8-1903928eb118 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:23:28.231806 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 85/167] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jul 25 05:23:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 05:23:28.232945 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-624a8815-73cc-4385-a35e-9d12c4f49143 None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:23:28.320087 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d84cfb1-33b6-4848-94d8-1903928eb118 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:23:28.351730 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:23:28.376651 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3792f2f9-1fd5-41f2-b126-8f51753686ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:23:28.377798 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3792f2f9-1fd5-41f2-b126-8f51753686ef None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:23:28.378233 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3792f2f9-1fd5-41f2-b126-8f51753686ef None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:23:28.378676 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3792f2f9-1fd5-41f2-b126-8f51753686ef None admin] PATH_INFO: `/v3/roles/member` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:23:28.390339 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3792f2f9-1fd5-41f2-b126-8f51753686ef None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 05:23:28.398140 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application [None req-3792f2f9-1fd5-41f2-b126-8f51753686ef None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 25 05:23:28.398140 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:23:28.398140 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:23:28.398140 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:23:28.398140 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:23:28.398140 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:23:28.398140 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:23:28.398140 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:23:28.398140 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:23:28.398140 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:23:28.398140 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:23:28.398140 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:23:28.398140 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 05:23:28.398140 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 05:23:28.398140 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 05:23:28.398140 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 05:23:28.399288 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 05:23:28.399288 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 05:23:28.399288 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 05:23:28.399288 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 05:23:28.399288 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:23:28.399288 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:23:28.399288 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:23:28.399288 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:23:28.399288 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:23:28.399288 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:23:28.399288 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:23:28.399288 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application with Lock( Jul 25 05:23:28.399288 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:23:28.399288 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:23:28.399288 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:23:28.399288 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:23:28.400087 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:23:28.400087 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:23:28.400087 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:23:28.400087 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:23:28.400087 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 05:23:28.400087 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 05:23:28.400087 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 05:23:28.400087 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 05:23:28.400087 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 05:23:28.400087 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 05:23:28.400087 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 25 05:23:28.400087 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Jul 25 05:23:28.400087 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 83/168] 10.222.0.33 () {56 vars in 1099 bytes} [Mon Jul 25 05:23:28 2022] GET /identity/v3/roles/member => generated 84 bytes in 291 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:23:28.400087 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3792f2f9-1fd5-41f2-b126-8f51753686ef None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:23:28.410019 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-624a8815-73cc-4385-a35e-9d12c4f49143 None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:23:28.424832 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:23:28.464884 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d46e907-c7cb-4ddb-a7c5-7a04f4a1f930 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:23:28.466402 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d46e907-c7cb-4ddb-a7c5-7a04f4a1f930 None admin] REQUEST_METHOD: `PUT` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:23:28.467008 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d46e907-c7cb-4ddb-a7c5-7a04f4a1f930 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:23:28.467761 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d46e907-c7cb-4ddb-a7c5-7a04f4a1f930 None admin] PATH_INFO: `/v3/projects/4b840bd0540b4512b180fc68fd41cedc/users/6fd6f234e1a54a4480852ab2c80c37c3/roles/be79956018b14baba7a021e2adf27084` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:23:28.472378 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d46e907-c7cb-4ddb-a7c5-7a04f4a1f930 None admin] RBAC: Authorizing `identity:create_grant(project_id=4b840bd0540b4512b180fc68fd41cedc, user_id=6fd6f234e1a54a4480852ab2c80c37c3, role_id=be79956018b14baba7a021e2adf27084)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:23:28.474071 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d46e907-c7cb-4ddb-a7c5-7a04f4a1f930 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:23:28.496447 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 86/169] 10.222.0.33 () {58 vars in 1439 bytes} [Mon Jul 25 05:23:28 2022] PUT /identity/v3/projects/4b840bd0540b4512b180fc68fd41cedc/users/6fd6f234e1a54a4480852ab2c80c37c3/roles/be79956018b14baba7a021e2adf27084 => generated 0 bytes in 265 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 05:23:28.498028 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0250d84e-88fc-46d0-bdf3-c05bdfa20544 None None] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:23:28.498470 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0250d84e-88fc-46d0-bdf3-c05bdfa20544 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:23:28.498919 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0250d84e-88fc-46d0-bdf3-c05bdfa20544 None None] PATH_INFO: `/` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:23:28.501290 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 87/170] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jul 25 05:23:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:23:28.502304 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0250d84e-88fc-46d0-bdf3-c05bdfa20544 None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:23:28.520930 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3792f2f9-1fd5-41f2-b126-8f51753686ef None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:23:28.568911 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9cb64df-7c09-43f3-a561-9d4902abc7df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:23:28.570140 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9cb64df-7c09-43f3-a561-9d4902abc7df None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:23:28.570656 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9cb64df-7c09-43f3-a561-9d4902abc7df None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:23:28.571207 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9cb64df-7c09-43f3-a561-9d4902abc7df None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:23:28.585344 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9cb64df-7c09-43f3-a561-9d4902abc7df None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 05:23:28.594843 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application [None req-f9cb64df-7c09-43f3-a561-9d4902abc7df None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 25 05:23:28.594843 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:23:28.594843 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:23:28.594843 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:23:28.594843 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:23:28.594843 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:23:28.594843 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:23:28.594843 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:23:28.594843 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:23:28.594843 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:23:28.594843 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:23:28.594843 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:23:28.594843 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 05:23:28.594843 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 05:23:28.594843 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 05:23:28.594843 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 05:23:28.595763 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 05:23:28.595763 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 05:23:28.595763 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 05:23:28.595763 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 05:23:28.595763 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:23:28.595763 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:23:28.595763 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:23:28.595763 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:23:28.595763 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:23:28.595763 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:23:28.595763 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:23:28.595763 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application with Lock( Jul 25 05:23:28.595763 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:23:28.595763 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:23:28.595763 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:23:28.595763 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:23:28.596544 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:23:28.596544 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:23:28.596544 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:23:28.596544 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:23:28.596544 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 05:23:28.596544 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 05:23:28.596544 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 05:23:28.596544 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 05:23:28.596544 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 05:23:28.596544 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 05:23:28.596544 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 25 05:23:28.596544 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Jul 25 05:23:28.599260 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 84/171] 10.222.0.33 () {56 vars in 1114 bytes} [Mon Jul 25 05:23:28 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:23:28.606619 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63d368b-d442-4e18-9b67-6a8c849a02ce None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:23:28.607125 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63d368b-d442-4e18-9b67-6a8c849a02ce None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:23:28.607648 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63d368b-d442-4e18-9b67-6a8c849a02ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:23:28.629152 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-e63d368b-d442-4e18-9b67-6a8c849a02ce None None] MFA Rules not processed for user `1eae540242d94c10b0ad33960a855f08`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:23:28.630725 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-e63d368b-d442-4e18-9b67-6a8c849a02ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:23:28.631335 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0250d84e-88fc-46d0-bdf3-c05bdfa20544 None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:23:28.658340 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:23:28.677645 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1813b734-1006-417b-bdfc-241d27780575 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:23:28.678342 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1813b734-1006-417b-bdfc-241d27780575 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:23:28.678627 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1813b734-1006-417b-bdfc-241d27780575 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:23:28.678911 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1813b734-1006-417b-bdfc-241d27780575 None admin] PATH_INFO: `/v3/roles` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:23:28.680013 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1813b734-1006-417b-bdfc-241d27780575 None admin] RBAC: Adding query filter params (name=member) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:23:28.680325 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1813b734-1006-417b-bdfc-241d27780575 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:23:28.682336 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1813b734-1006-417b-bdfc-241d27780575 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:23:28.693398 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 88/172] 10.222.0.33 () {56 vars in 1101 bytes} [Mon Jul 25 05:23:28 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:23:28.694081 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1813b734-1006-417b-bdfc-241d27780575 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:23:28.763622 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 85/173] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jul 25 05:23:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 05:23:28.764383 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e63d368b-d442-4e18-9b67-6a8c849a02ce None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:23:28.780419 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:23:28.831983 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1813b734-1006-417b-bdfc-241d27780575 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:23:28.877374 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d30574b8-9665-43e6-8f19-6c8d4f83a272 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:23:28.878554 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30574b8-9665-43e6-8f19-6c8d4f83a272 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:23:28.879222 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30574b8-9665-43e6-8f19-6c8d4f83a272 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:23:28.879651 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30574b8-9665-43e6-8f19-6c8d4f83a272 None admin] PATH_INFO: `/v3/roles` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:23:28.881236 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d30574b8-9665-43e6-8f19-6c8d4f83a272 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:23:28.881709 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d30574b8-9665-43e6-8f19-6c8d4f83a272 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:23:28.884098 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d30574b8-9665-43e6-8f19-6c8d4f83a272 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:23:28.885869 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e63d368b-d442-4e18-9b67-6a8c849a02ce None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:23:28.896592 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 89/174] 10.222.0.33 () {56 vars in 1111 bytes} [Mon Jul 25 05:23:28 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:23:28.897663 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d30574b8-9665-43e6-8f19-6c8d4f83a272 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:23:28.926131 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af97017-5bcd-44cd-a5c3-dcac1746c020 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:23:28.927076 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af97017-5bcd-44cd-a5c3-dcac1746c020 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:23:28.927358 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af97017-5bcd-44cd-a5c3-dcac1746c020 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:23:28.927683 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af97017-5bcd-44cd-a5c3-dcac1746c020 None admin] PATH_INFO: `/v3/users/c25734c1f07644de9355e54cfd5f65cc` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:23:28.928977 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8af97017-5bcd-44cd-a5c3-dcac1746c020 None admin] RBAC: Authorizing `identity:get_user(user_id=c25734c1f07644de9355e54cfd5f65cc)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:23:28.930458 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8af97017-5bcd-44cd-a5c3-dcac1746c020 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:23:28.932186 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 86/175] 10.222.0.33 () {56 vars in 1177 bytes} [Mon Jul 25 05:23:28 2022] GET /identity/v3/users/c25734c1f07644de9355e54cfd5f65cc => generated 275 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:23:28.933104 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af97017-5bcd-44cd-a5c3-dcac1746c020 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:23:28.962026 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:23:29.020344 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:23:29.041205 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d30574b8-9665-43e6-8f19-6c8d4f83a272 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:23:29.053082 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af97017-5bcd-44cd-a5c3-dcac1746c020 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:23:29.081120 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07137dbe-5059-4d92-b25e-3742fe3e2d84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:23:29.082035 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07137dbe-5059-4d92-b25e-3742fe3e2d84 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:23:29.082393 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07137dbe-5059-4d92-b25e-3742fe3e2d84 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:23:29.082732 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07137dbe-5059-4d92-b25e-3742fe3e2d84 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:23:29.094387 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07137dbe-5059-4d92-b25e-3742fe3e2d84 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 05:23:29.101195 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-792c7a64-78e2-4de5-899d-71fd8d46f918 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:23:29.102286 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792c7a64-78e2-4de5-899d-71fd8d46f918 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:23:29.102750 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792c7a64-78e2-4de5-899d-71fd8d46f918 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:23:29.102902 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application [None req-07137dbe-5059-4d92-b25e-3742fe3e2d84 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 05:23:29.102902 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:23:29.102902 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:23:29.102902 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:23:29.102902 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:23:29.102902 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:23:29.102902 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:23:29.102902 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:23:29.102902 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:23:29.102902 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:23:29.102902 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:23:29.102902 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:23:29.102902 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 05:23:29.102902 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 05:23:29.102902 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 05:23:29.102902 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 05:23:29.103762 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 05:23:29.103762 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 05:23:29.103762 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 05:23:29.103762 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 05:23:29.103762 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:23:29.103762 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:23:29.103762 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:23:29.103762 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:23:29.103762 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:23:29.103762 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:23:29.103762 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:23:29.103762 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application with Lock( Jul 25 05:23:29.103762 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:23:29.103762 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:23:29.103762 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:23:29.103762 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:23:29.104565 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:23:29.104565 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:23:29.104565 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:23:29.104565 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:23:29.104565 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 05:23:29.104565 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 05:23:29.104565 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 05:23:29.104565 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 05:23:29.104565 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 05:23:29.104565 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 05:23:29.104565 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 05:23:29.104565 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Jul 25 05:23:29.104565 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 90/176] 10.222.0.33 () {56 vars in 1096 bytes} [Mon Jul 25 05:23:28 2022] GET /identity/v3/roles/admin => generated 83 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:23:29.106035 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07137dbe-5059-4d92-b25e-3742fe3e2d84 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:23:29.106340 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792c7a64-78e2-4de5-899d-71fd8d46f918 None admin] PATH_INFO: `/v3/users/c25734c1f07644de9355e54cfd5f65cc` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:23:29.107777 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-792c7a64-78e2-4de5-899d-71fd8d46f918 None admin] RBAC: Authorizing `identity:get_user(user_id=c25734c1f07644de9355e54cfd5f65cc)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:23:29.109526 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-792c7a64-78e2-4de5-899d-71fd8d46f918 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:23:29.112745 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 87/177] 10.222.0.33 () {56 vars in 1177 bytes} [Mon Jul 25 05:23:28 2022] GET /identity/v3/users/c25734c1f07644de9355e54cfd5f65cc => generated 275 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:23:29.114218 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68876b5c-f102-4702-a3ea-ffa9c2764a81 None None] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:23:29.114688 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68876b5c-f102-4702-a3ea-ffa9c2764a81 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:23:29.115138 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68876b5c-f102-4702-a3ea-ffa9c2764a81 None None] PATH_INFO: `/` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:23:29.116569 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 88/178] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jul 25 05:23:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:23:29.117462 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68876b5c-f102-4702-a3ea-ffa9c2764a81 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:23:29.238348 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07137dbe-5059-4d92-b25e-3742fe3e2d84 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:23:29.242230 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:23:29.248441 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68876b5c-f102-4702-a3ea-ffa9c2764a81 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:23:29.287422 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc55d52-22d7-4f6a-a425-2452ec42406f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:23:29.288089 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc55d52-22d7-4f6a-a425-2452ec42406f None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:23:29.288380 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc55d52-22d7-4f6a-a425-2452ec42406f None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:23:29.288662 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc55d52-22d7-4f6a-a425-2452ec42406f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:23:29.291663 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea1bbb87-adae-490a-b45f-aa2d145c9490 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:23:29.292636 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1bbb87-adae-490a-b45f-aa2d145c9490 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:23:29.293147 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1bbb87-adae-490a-b45f-aa2d145c9490 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:23:29.293619 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1bbb87-adae-490a-b45f-aa2d145c9490 None admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:23:29.294615 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea1bbb87-adae-490a-b45f-aa2d145c9490 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:23:29.295115 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea1bbb87-adae-490a-b45f-aa2d145c9490 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:23:29.296825 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea1bbb87-adae-490a-b45f-aa2d145c9490 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:23:29.306194 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 89/179] 10.222.0.33 () {56 vars in 1099 bytes} [Mon Jul 25 05:23:29 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:23:29.307812 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a8745f-143a-437c-a199-7bb8c4e23436 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:23:29.308312 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a8745f-143a-437c-a199-7bb8c4e23436 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:23:29.308826 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a8745f-143a-437c-a199-7bb8c4e23436 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:23:29.326158 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-b0a8745f-143a-437c-a199-7bb8c4e23436 None None] MFA Rules not processed for user `1eae540242d94c10b0ad33960a855f08`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:23:29.328061 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-b0a8745f-143a-437c-a199-7bb8c4e23436 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:23:29.330377 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:23:29.367192 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bc55d52-22d7-4f6a-a425-2452ec42406f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:23:29.368660 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bc55d52-22d7-4f6a-a425-2452ec42406f None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:23:29.404756 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 90/180] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jul 25 05:23:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 05:23:29.406018 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a8745f-143a-437c-a199-7bb8c4e23436 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:23:29.475922 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 91/181] 10.222.0.33 () {58 vars in 1282 bytes} [Mon Jul 25 05:23:29 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 371 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 25 05:23:29.477803 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1009b1-6a0d-4bf5-8074-23222af3c608 None None] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:23:29.477803 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1009b1-6a0d-4bf5-8074-23222af3c608 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:23:29.479578 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1009b1-6a0d-4bf5-8074-23222af3c608 None None] PATH_INFO: `/` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:23:29.481063 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 92/182] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jul 25 05:23:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:23:29.482174 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee1009b1-6a0d-4bf5-8074-23222af3c608 None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:23:29.486481 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:23:29.521555 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a8745f-143a-437c-a199-7bb8c4e23436 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:23:29.558923 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c031048-5d92-4927-8a83-0043b8c258ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:23:29.559577 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c031048-5d92-4927-8a83-0043b8c258ce None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:23:29.559865 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c031048-5d92-4927-8a83-0043b8c258ce None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:23:29.560147 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c031048-5d92-4927-8a83-0043b8c258ce None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:23:29.569822 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:23:29.594447 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee1009b1-6a0d-4bf5-8074-23222af3c608 None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:23:29.630136 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e4eb7ce-ed0a-4587-b004-d7927ac52bab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:23:29.631009 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4eb7ce-ed0a-4587-b004-d7927ac52bab None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:23:29.631299 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4eb7ce-ed0a-4587-b004-d7927ac52bab None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:23:29.631577 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4eb7ce-ed0a-4587-b004-d7927ac52bab None admin] PATH_INFO: `/v3/users/1c6dda23da0644c2b35c2b1ffc72a8f1` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:23:29.633141 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c031048-5d92-4927-8a83-0043b8c258ce None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:23:29.634692 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c031048-5d92-4927-8a83-0043b8c258ce None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:23:29.646376 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e4eb7ce-ed0a-4587-b004-d7927ac52bab None admin] RBAC: Authorizing `identity:get_user(user_id=1c6dda23da0644c2b35c2b1ffc72a8f1)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:23:29.647948 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e4eb7ce-ed0a-4587-b004-d7927ac52bab None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:23:29.649862 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 93/183] 10.222.0.33 () {56 vars in 1177 bytes} [Mon Jul 25 05:23:29 2022] GET /identity/v3/users/1c6dda23da0644c2b35c2b1ffc72a8f1 => generated 283 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:23:29.650445 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e4eb7ce-ed0a-4587-b004-d7927ac52bab None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:23:29.716443 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:23:29.745607 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 91/184] 10.222.0.33 () {58 vars in 1282 bytes} [Mon Jul 25 05:23:29 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 340 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 25 05:23:29.748989 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c031048-5d92-4927-8a83-0043b8c258ce None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:23:29.761602 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e4eb7ce-ed0a-4587-b004-d7927ac52bab None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:23:29.788601 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:23:29.798903 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a6ef1a5-3a8c-4f0a-b7f5-34085574aae7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:23:29.799877 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6ef1a5-3a8c-4f0a-b7f5-34085574aae7 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:23:29.800165 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6ef1a5-3a8c-4f0a-b7f5-34085574aae7 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:23:29.800441 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6ef1a5-3a8c-4f0a-b7f5-34085574aae7 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:23:29.817034 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a6ef1a5-3a8c-4f0a-b7f5-34085574aae7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 05:23:29.827045 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application [None req-2a6ef1a5-3a8c-4f0a-b7f5-34085574aae7 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 25 05:23:29.827045 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:23:29.827045 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:23:29.827045 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:23:29.827045 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:23:29.827045 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:23:29.827045 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:23:29.827045 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:23:29.827045 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:23:29.827045 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:23:29.827045 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:23:29.827045 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:23:29.827045 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 05:23:29.827045 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 05:23:29.827045 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 05:23:29.827045 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 05:23:29.828622 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 05:23:29.828622 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 05:23:29.828622 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 05:23:29.828622 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 05:23:29.828622 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:23:29.828622 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:23:29.828622 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:23:29.828622 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:23:29.828622 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:23:29.828622 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:23:29.828622 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:23:29.828622 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application with Lock( Jul 25 05:23:29.828622 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:23:29.828622 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:23:29.828622 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:23:29.828622 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:23:29.829464 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:23:29.829464 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:23:29.829464 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:23:29.829464 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:23:29.829464 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 05:23:29.829464 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 05:23:29.829464 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 05:23:29.829464 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 05:23:29.829464 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 05:23:29.829464 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 05:23:29.829464 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 25 05:23:29.829464 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Jul 25 05:23:29.835340 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 94/185] 10.222.0.33 () {56 vars in 1114 bytes} [Mon Jul 25 05:23:29 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:23:29.835340 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb026ca-bbdd-453d-bd4f-3feb94d18b2b None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:23:29.835340 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb026ca-bbdd-453d-bd4f-3feb94d18b2b None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:23:29.835340 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb026ca-bbdd-453d-bd4f-3feb94d18b2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:23:29.853187 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c031048-5d92-4927-8a83-0043b8c258ce None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:23:29.854445 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-0eb026ca-bbdd-453d-bd4f-3feb94d18b2b None None] MFA Rules not processed for user `1eae540242d94c10b0ad33960a855f08`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:23:29.856724 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-0eb026ca-bbdd-453d-bd4f-3feb94d18b2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:23:29.892956 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d24b31-8fe1-404b-b71d-fac93bb5a38c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:23:29.893852 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d24b31-8fe1-404b-b71d-fac93bb5a38c None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:23:29.894134 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d24b31-8fe1-404b-b71d-fac93bb5a38c None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:23:29.894409 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d24b31-8fe1-404b-b71d-fac93bb5a38c None admin] PATH_INFO: `/v3/projects/863d4de6ca1540baa4c4aef54e727bae` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:23:29.895697 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03d24b31-8fe1-404b-b71d-fac93bb5a38c None admin] RBAC: Authorizing `identity:get_project(project_id=863d4de6ca1540baa4c4aef54e727bae)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:23:29.897216 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03d24b31-8fe1-404b-b71d-fac93bb5a38c None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:23:29.899184 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 92/186] 10.222.0.33 () {56 vars in 1186 bytes} [Mon Jul 25 05:23:29 2022] GET /identity/v3/projects/863d4de6ca1540baa4c4aef54e727bae => generated 312 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:23:29.899931 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d24b31-8fe1-404b-b71d-fac93bb5a38c None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:23:29.939431 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 95/187] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jul 25 05:23:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 05:23:29.940970 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb026ca-bbdd-453d-bd4f-3feb94d18b2b None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:23:30.008603 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:23:30.013702 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d24b31-8fe1-404b-b71d-fac93bb5a38c None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:23:30.022198 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:23:30.052036 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2680f081-2d37-4bd1-894a-48fc538d7728 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:23:30.053832 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2680f081-2d37-4bd1-894a-48fc538d7728 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:23:30.054411 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2680f081-2d37-4bd1-894a-48fc538d7728 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:23:30.054411 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2680f081-2d37-4bd1-894a-48fc538d7728 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:23:30.059676 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb026ca-bbdd-453d-bd4f-3feb94d18b2b None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:23:30.096869 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ca2a90-3306-4392-8887-a108f252c12c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:23:30.097825 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ca2a90-3306-4392-8887-a108f252c12c None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:23:30.098459 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ca2a90-3306-4392-8887-a108f252c12c None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:23:30.098459 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ca2a90-3306-4392-8887-a108f252c12c None admin] PATH_INFO: `/v3/projects/4b840bd0540b4512b180fc68fd41cedc` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:23:30.099762 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16ca2a90-3306-4392-8887-a108f252c12c None admin] RBAC: Authorizing `identity:get_project(project_id=4b840bd0540b4512b180fc68fd41cedc)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:23:30.101355 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16ca2a90-3306-4392-8887-a108f252c12c None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:23:30.103075 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 96/188] 10.222.0.33 () {56 vars in 1186 bytes} [Mon Jul 25 05:23:29 2022] GET /identity/v3/projects/4b840bd0540b4512b180fc68fd41cedc => generated 298 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:23:30.108090 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ca2a90-3306-4392-8887-a108f252c12c None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:23:30.132272 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2680f081-2d37-4bd1-894a-48fc538d7728 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:23:30.133792 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2680f081-2d37-4bd1-894a-48fc538d7728 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:23:30.212227 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ca2a90-3306-4392-8887-a108f252c12c None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:23:30.236706 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 93/189] 10.222.0.33 () {58 vars in 1282 bytes} [Mon Jul 25 05:23:29 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 337 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 25 05:23:30.238637 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:23:30.239829 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2680f081-2d37-4bd1-894a-48fc538d7728 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:23:30.250343 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:23:30.255390 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98bc7a16-0597-4f85-93e9-4b50543e9d1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:23:30.256065 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98bc7a16-0597-4f85-93e9-4b50543e9d1a None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:23:30.256365 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98bc7a16-0597-4f85-93e9-4b50543e9d1a None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:23:30.256628 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98bc7a16-0597-4f85-93e9-4b50543e9d1a None admin] PATH_INFO: `/v3/roles` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:23:30.257408 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98bc7a16-0597-4f85-93e9-4b50543e9d1a None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:23:30.257716 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98bc7a16-0597-4f85-93e9-4b50543e9d1a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:23:30.259083 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98bc7a16-0597-4f85-93e9-4b50543e9d1a None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:23:30.267967 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 97/190] 10.222.0.33 () {56 vars in 1111 bytes} [Mon Jul 25 05:23:30 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:23:30.268666 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98bc7a16-0597-4f85-93e9-4b50543e9d1a None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:23:30.355034 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2680f081-2d37-4bd1-894a-48fc538d7728 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:23:30.379958 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98bc7a16-0597-4f85-93e9-4b50543e9d1a None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:23:30.393649 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-275b0f47-8a76-4ef5-99ba-b0745d795691 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:23:30.394420 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275b0f47-8a76-4ef5-99ba-b0745d795691 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:23:30.394705 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275b0f47-8a76-4ef5-99ba-b0745d795691 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:23:30.394987 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275b0f47-8a76-4ef5-99ba-b0745d795691 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:23:30.395824 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-275b0f47-8a76-4ef5-99ba-b0745d795691 None admin] RBAC: Adding query filter params (role.id=5074e74e35f54d4b9bb415c34999a82a, scope.project.id=863d4de6ca1540baa4c4aef54e727bae, user.id=c25734c1f07644de9355e54cfd5f65cc) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:23:30.396134 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-275b0f47-8a76-4ef5-99ba-b0745d795691 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:23:30.397825 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-275b0f47-8a76-4ef5-99ba-b0745d795691 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:23:30.404155 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 94/191] 10.222.0.33 () {56 vars in 1374 bytes} [Mon Jul 25 05:23:30 2022] GET /identity/v3/role_assignments?user.id=c25734c1f07644de9355e54cfd5f65cc&scope.project.id=863d4de6ca1540baa4c4aef54e727bae&role.id=5074e74e35f54d4b9bb415c34999a82a => generated 608 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:23:30.404871 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-275b0f47-8a76-4ef5-99ba-b0745d795691 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:23:30.418260 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c793b69c-0baf-4ac6-9468-5697d6c9b03e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:23:30.420270 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c793b69c-0baf-4ac6-9468-5697d6c9b03e None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:23:30.420835 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c793b69c-0baf-4ac6-9468-5697d6c9b03e None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:23:30.421881 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c793b69c-0baf-4ac6-9468-5697d6c9b03e None admin] PATH_INFO: `/v3/domains/default` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:23:30.424408 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c793b69c-0baf-4ac6-9468-5697d6c9b03e None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:23:30.426822 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c793b69c-0baf-4ac6-9468-5697d6c9b03e None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:23:30.429496 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 98/192] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Jul 25 05:23:30 2022] GET /identity/v3/domains/default => generated 199 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:23:30.430591 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c793b69c-0baf-4ac6-9468-5697d6c9b03e None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:23:30.495306 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:23:30.512500 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:23:30.538737 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-275b0f47-8a76-4ef5-99ba-b0745d795691 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:23:30.583802 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be66ff1-b94f-47a2-9608-0df6792cd15b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:23:30.584465 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be66ff1-b94f-47a2-9608-0df6792cd15b None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:23:30.584750 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be66ff1-b94f-47a2-9608-0df6792cd15b None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:23:30.585067 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be66ff1-b94f-47a2-9608-0df6792cd15b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:23:30.586060 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3be66ff1-b94f-47a2-9608-0df6792cd15b None admin] RBAC: Adding query filter params (role.id=02aba1f7cbdf4d6595e18af7fb5727e0, scope.project.id=4b840bd0540b4512b180fc68fd41cedc, user.id=c25734c1f07644de9355e54cfd5f65cc) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:23:30.586377 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3be66ff1-b94f-47a2-9608-0df6792cd15b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:23:30.587817 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3be66ff1-b94f-47a2-9608-0df6792cd15b None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:23:30.589379 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c793b69c-0baf-4ac6-9468-5697d6c9b03e None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:23:30.594788 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 95/193] 10.222.0.33 () {56 vars in 1374 bytes} [Mon Jul 25 05:23:30 2022] GET /identity/v3/role_assignments?user.id=c25734c1f07644de9355e54cfd5f65cc&scope.project.id=4b840bd0540b4512b180fc68fd41cedc&role.id=02aba1f7cbdf4d6595e18af7fb5727e0 => generated 608 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:23:30.602262 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be66ff1-b94f-47a2-9608-0df6792cd15b None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:23:30.635382 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08bac128-3c94-4801-84fa-96f46cf8c4f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:23:30.636599 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08bac128-3c94-4801-84fa-96f46cf8c4f2 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:23:30.637188 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08bac128-3c94-4801-84fa-96f46cf8c4f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:23:30.637688 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08bac128-3c94-4801-84fa-96f46cf8c4f2 None admin] PATH_INFO: `/v3/projects/ec1ed227a23f453cab64e15036af0c60` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:23:30.639188 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08bac128-3c94-4801-84fa-96f46cf8c4f2 None admin] RBAC: Authorizing `identity:get_project(project_id=ec1ed227a23f453cab64e15036af0c60)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:23:30.642377 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08bac128-3c94-4801-84fa-96f46cf8c4f2 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:23:30.644928 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 99/194] 10.222.0.33 () {56 vars in 1186 bytes} [Mon Jul 25 05:23:30 2022] GET /identity/v3/projects/ec1ed227a23f453cab64e15036af0c60 => generated 302 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:23:30.646020 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08bac128-3c94-4801-84fa-96f46cf8c4f2 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:23:30.759221 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be66ff1-b94f-47a2-9608-0df6792cd15b None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:23:30.793475 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08bac128-3c94-4801-84fa-96f46cf8c4f2 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:23:30.808634 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3556eab3-2ef3-493e-b28e-e97976cc698c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:23:30.809698 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3556eab3-2ef3-493e-b28e-e97976cc698c None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:23:30.810058 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3556eab3-2ef3-493e-b28e-e97976cc698c None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:23:30.810409 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3556eab3-2ef3-493e-b28e-e97976cc698c None admin] PATH_INFO: `/v3/users/1c6dda23da0644c2b35c2b1ffc72a8f1` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:23:30.811799 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3556eab3-2ef3-493e-b28e-e97976cc698c None admin] RBAC: Authorizing `identity:get_user(user_id=1c6dda23da0644c2b35c2b1ffc72a8f1)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:23:30.813895 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3556eab3-2ef3-493e-b28e-e97976cc698c None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:23:30.816010 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 96/195] 10.222.0.33 () {56 vars in 1177 bytes} [Mon Jul 25 05:23:30 2022] GET /identity/v3/users/1c6dda23da0644c2b35c2b1ffc72a8f1 => generated 283 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:23:30.816415 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3556eab3-2ef3-493e-b28e-e97976cc698c None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:23:30.824302 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:23:30.826233 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:23:30.855668 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efcd0f02-106a-4688-acdc-41471fba5e95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:23:30.855668 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efcd0f02-106a-4688-acdc-41471fba5e95 None admin] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:23:30.855926 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efcd0f02-106a-4688-acdc-41471fba5e95 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:23:30.856021 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efcd0f02-106a-4688-acdc-41471fba5e95 None admin] PATH_INFO: `/v3/users` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:23:30.857560 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efcd0f02-106a-4688-acdc-41471fba5e95 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:23:30.859660 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efcd0f02-106a-4688-acdc-41471fba5e95 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:23:30.881212 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 100/196] 10.222.0.33 () {60 vars in 1132 bytes} [Mon Jul 25 05:23:30 2022] POST /identity/v3/users => generated 297 bytes in 235 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 05:23:30.882051 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efcd0f02-106a-4688-acdc-41471fba5e95 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:23:30.951489 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3556eab3-2ef3-493e-b28e-e97976cc698c None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:23:30.992671 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fecf5c4-9d82-4310-af9d-9e5996da7592 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:23:30.993912 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fecf5c4-9d82-4310-af9d-9e5996da7592 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:23:30.994209 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fecf5c4-9d82-4310-af9d-9e5996da7592 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:23:30.994662 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fecf5c4-9d82-4310-af9d-9e5996da7592 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:23:30.995517 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fecf5c4-9d82-4310-af9d-9e5996da7592 None admin] RBAC: Adding query filter params (role.id=39e5c666285a42fe8fc2c3afca6260d1, scope.project.id=ec1ed227a23f453cab64e15036af0c60, user.id=1c6dda23da0644c2b35c2b1ffc72a8f1) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:23:30.995952 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fecf5c4-9d82-4310-af9d-9e5996da7592 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:23:30.997356 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fecf5c4-9d82-4310-af9d-9e5996da7592 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:23:31.004680 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 97/197] 10.222.0.33 () {56 vars in 1374 bytes} [Mon Jul 25 05:23:30 2022] GET /identity/v3/role_assignments?user.id=1c6dda23da0644c2b35c2b1ffc72a8f1&scope.project.id=ec1ed227a23f453cab64e15036af0c60&role.id=39e5c666285a42fe8fc2c3afca6260d1 => generated 259 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:23:31.008147 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efcd0f02-106a-4688-acdc-41471fba5e95 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:23:31.014505 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a764b1-3876-4bbe-b58c-b38264a2e355 None None] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:23:31.015340 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a764b1-3876-4bbe-b58c-b38264a2e355 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:23:31.015837 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a764b1-3876-4bbe-b58c-b38264a2e355 None None] PATH_INFO: `/` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:23:31.023476 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 98/198] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jul 25 05:23:31 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:23:31.027256 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8daf7282-2086-48dd-bbf3-82bb8e2884bf None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:23:31.027881 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8daf7282-2086-48dd-bbf3-82bb8e2884bf None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:23:31.028675 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8daf7282-2086-48dd-bbf3-82bb8e2884bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:23:31.052481 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-8daf7282-2086-48dd-bbf3-82bb8e2884bf None None] MFA Rules not processed for user `1eae540242d94c10b0ad33960a855f08`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:23:31.053798 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-8daf7282-2086-48dd-bbf3-82bb8e2884bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:23:31.066991 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f577e6-9b19-415d-a582-e7404c9ae785 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:23:31.068135 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f577e6-9b19-415d-a582-e7404c9ae785 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:23:31.068723 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f577e6-9b19-415d-a582-e7404c9ae785 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:23:31.069282 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f577e6-9b19-415d-a582-e7404c9ae785 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:23:31.133443 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:23:31.141510 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:23:31.152054 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 99/199] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jul 25 05:23:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 05:23:31.165542 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83f577e6-9b19-415d-a582-e7404c9ae785 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:23:31.167274 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83f577e6-9b19-415d-a582-e7404c9ae785 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:23:31.176947 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8daf7282-2086-48dd-bbf3-82bb8e2884bf None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:23:31.292179 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 101/200] 10.222.0.33 () {58 vars in 1282 bytes} [Mon Jul 25 05:23:30 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 410 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 25 05:23:31.299073 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f577e6-9b19-415d-a582-e7404c9ae785 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:23:31.306474 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8daf7282-2086-48dd-bbf3-82bb8e2884bf None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:23:31.357812 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d12c4e3-a646-427c-94af-4b5fada2aa7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:23:31.359733 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d12c4e3-a646-427c-94af-4b5fada2aa7f None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:23:31.360035 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d12c4e3-a646-427c-94af-4b5fada2aa7f None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:23:31.360320 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d12c4e3-a646-427c-94af-4b5fada2aa7f None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:23:31.376116 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d12c4e3-a646-427c-94af-4b5fada2aa7f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 05:23:31.386820 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application [None req-1d12c4e3-a646-427c-94af-4b5fada2aa7f None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 25 05:23:31.386820 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:23:31.386820 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:23:31.386820 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:23:31.386820 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:23:31.386820 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:23:31.386820 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:23:31.386820 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:23:31.386820 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:23:31.386820 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:23:31.386820 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:23:31.386820 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:23:31.386820 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 05:23:31.386820 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 05:23:31.386820 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 05:23:31.386820 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 05:23:31.387739 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 05:23:31.387739 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 05:23:31.387739 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 05:23:31.387739 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 05:23:31.387739 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:23:31.387739 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:23:31.387739 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:23:31.387739 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:23:31.387739 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:23:31.387739 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:23:31.387739 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:23:31.387739 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application with Lock( Jul 25 05:23:31.387739 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:23:31.387739 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:23:31.387739 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:23:31.387739 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:23:31.388530 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:23:31.388530 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:23:31.388530 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:23:31.388530 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:23:31.388530 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 05:23:31.388530 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 05:23:31.388530 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 05:23:31.388530 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 05:23:31.388530 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 05:23:31.388530 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 05:23:31.388530 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 25 05:23:31.388530 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Jul 25 05:23:31.388530 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 100/201] 10.222.0.33 () {56 vars in 1099 bytes} [Mon Jul 25 05:23:31 2022] GET /identity/v3/roles/reader => generated 84 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 25 05:23:31.395224 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d12c4e3-a646-427c-94af-4b5fada2aa7f None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:23:31.396283 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:23:31.425658 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f577e6-9b19-415d-a582-e7404c9ae785 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:23:31.467343 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e4ea597-94e7-4ff4-8cc2-ee38e0bcd3aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:23:31.469146 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4ea597-94e7-4ff4-8cc2-ee38e0bcd3aa None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:23:31.469619 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4ea597-94e7-4ff4-8cc2-ee38e0bcd3aa None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:23:31.470077 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4ea597-94e7-4ff4-8cc2-ee38e0bcd3aa None admin] PATH_INFO: `/v3/projects/ec1ed227a23f453cab64e15036af0c60` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:23:31.471553 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e4ea597-94e7-4ff4-8cc2-ee38e0bcd3aa None admin] RBAC: Authorizing `identity:get_project(project_id=ec1ed227a23f453cab64e15036af0c60)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:23:31.474089 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e4ea597-94e7-4ff4-8cc2-ee38e0bcd3aa None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:23:31.476029 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:23:31.477582 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 102/202] 10.222.0.33 () {56 vars in 1186 bytes} [Mon Jul 25 05:23:31 2022] GET /identity/v3/projects/ec1ed227a23f453cab64e15036af0c60 => generated 302 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:23:31.483667 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e4ea597-94e7-4ff4-8cc2-ee38e0bcd3aa None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:23:31.537377 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d12c4e3-a646-427c-94af-4b5fada2aa7f None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:23:31.602227 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e4ea597-94e7-4ff4-8cc2-ee38e0bcd3aa None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:23:31.609984 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b48ac6-3ab1-452e-8b23-257f6c00f87e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:23:31.610690 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b48ac6-3ab1-452e-8b23-257f6c00f87e None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:23:31.610976 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b48ac6-3ab1-452e-8b23-257f6c00f87e None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:23:31.611254 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b48ac6-3ab1-452e-8b23-257f6c00f87e None admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:23:31.612028 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70b48ac6-3ab1-452e-8b23-257f6c00f87e None admin] RBAC: Adding query filter params (name=reader) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:23:31.612351 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70b48ac6-3ab1-452e-8b23-257f6c00f87e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:23:31.613791 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70b48ac6-3ab1-452e-8b23-257f6c00f87e None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:23:31.623001 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 101/203] 10.222.0.33 () {56 vars in 1101 bytes} [Mon Jul 25 05:23:31 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 05:23:31.628614 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b48ac6-3ab1-452e-8b23-257f6c00f87e None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:23:31.647429 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c543c4d-f99d-4a46-ad5a-6136e1873b15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:23:31.648610 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c543c4d-f99d-4a46-ad5a-6136e1873b15 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:23:31.649117 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c543c4d-f99d-4a46-ad5a-6136e1873b15 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:23:31.649600 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c543c4d-f99d-4a46-ad5a-6136e1873b15 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:23:31.650629 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c543c4d-f99d-4a46-ad5a-6136e1873b15 None admin] RBAC: Adding query filter params (role.id=02aba1f7cbdf4d6595e18af7fb5727e0, scope.project.id=ec1ed227a23f453cab64e15036af0c60, user.id=1c6dda23da0644c2b35c2b1ffc72a8f1) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:23:31.651333 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c543c4d-f99d-4a46-ad5a-6136e1873b15 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:23:31.653166 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c543c4d-f99d-4a46-ad5a-6136e1873b15 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:23:31.660540 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 103/204] 10.222.0.33 () {56 vars in 1374 bytes} [Mon Jul 25 05:23:31 2022] GET /identity/v3/role_assignments?user.id=1c6dda23da0644c2b35c2b1ffc72a8f1&scope.project.id=ec1ed227a23f453cab64e15036af0c60&role.id=02aba1f7cbdf4d6595e18af7fb5727e0 => generated 259 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 05:23:31.748106 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b48ac6-3ab1-452e-8b23-257f6c00f87e None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:23:31.767862 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:23:31.792401 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7fd9e9-41a3-4a2a-8c72-c3910e0ce01b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:23:31.793326 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7fd9e9-41a3-4a2a-8c72-c3910e0ce01b None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:23:31.793615 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7fd9e9-41a3-4a2a-8c72-c3910e0ce01b None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:23:31.793902 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7fd9e9-41a3-4a2a-8c72-c3910e0ce01b None admin] PATH_INFO: `/v3/users/6fd6f234e1a54a4480852ab2c80c37c3` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:23:31.797755 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d7fd9e9-41a3-4a2a-8c72-c3910e0ce01b None admin] RBAC: Authorizing `identity:get_user(user_id=6fd6f234e1a54a4480852ab2c80c37c3)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:23:31.801094 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d7fd9e9-41a3-4a2a-8c72-c3910e0ce01b None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:23:31.801247 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 102/205] 10.222.0.33 () {56 vars in 1177 bytes} [Mon Jul 25 05:23:31 2022] GET /identity/v3/users/6fd6f234e1a54a4480852ab2c80c37c3 => generated 289 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 05:23:31.807011 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c543c4d-f99d-4a46-ad5a-6136e1873b15 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:23:31.828103 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:23:31.969605 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c543c4d-f99d-4a46-ad5a-6136e1873b15 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:23:32.019099 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e051b529-e2ee-4c90-9209-b55d8e498709 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:23:32.019760 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e051b529-e2ee-4c90-9209-b55d8e498709 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:23:32.020047 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e051b529-e2ee-4c90-9209-b55d8e498709 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:23:32.020326 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e051b529-e2ee-4c90-9209-b55d8e498709 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:23:32.108454 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e051b529-e2ee-4c90-9209-b55d8e498709 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:23:32.110062 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e051b529-e2ee-4c90-9209-b55d8e498709 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:23:32.123033 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:23:32.233157 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 104/206] 10.222.0.33 () {58 vars in 1282 bytes} [Mon Jul 25 05:23:31 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 426 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 25 05:23:32.241882 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7fd9e9-41a3-4a2a-8c72-c3910e0ce01b None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:23:32.365960 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7fd9e9-41a3-4a2a-8c72-c3910e0ce01b None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:23:32.408696 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc9d1f6-7393-4930-9cd6-54e6233f4ac9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:23:32.409609 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc9d1f6-7393-4930-9cd6-54e6233f4ac9 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:23:32.409898 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc9d1f6-7393-4930-9cd6-54e6233f4ac9 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:23:32.410176 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc9d1f6-7393-4930-9cd6-54e6233f4ac9 None admin] PATH_INFO: `/v3/projects/4b840bd0540b4512b180fc68fd41cedc` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:23:32.411435 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dc9d1f6-7393-4930-9cd6-54e6233f4ac9 None admin] RBAC: Authorizing `identity:get_project(project_id=4b840bd0540b4512b180fc68fd41cedc)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:23:32.412961 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dc9d1f6-7393-4930-9cd6-54e6233f4ac9 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:23:32.415019 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 103/207] 10.222.0.33 () {56 vars in 1186 bytes} [Mon Jul 25 05:23:32 2022] GET /identity/v3/projects/4b840bd0540b4512b180fc68fd41cedc => generated 298 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:23:32.421540 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e051b529-e2ee-4c90-9209-b55d8e498709 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:23:32.579431 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e051b529-e2ee-4c90-9209-b55d8e498709 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:23:32.590994 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:23:32.638028 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e650706-235b-4336-9e3b-da3e0af83d62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:23:32.638697 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e650706-235b-4336-9e3b-da3e0af83d62 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:23:32.638984 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e650706-235b-4336-9e3b-da3e0af83d62 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:23:32.639260 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e650706-235b-4336-9e3b-da3e0af83d62 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:23:32.640160 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e650706-235b-4336-9e3b-da3e0af83d62 None admin] RBAC: Adding query filter params (role.id=be79956018b14baba7a021e2adf27084, scope.project.id=4b840bd0540b4512b180fc68fd41cedc, user.id=6fd6f234e1a54a4480852ab2c80c37c3) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:23:32.640461 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e650706-235b-4336-9e3b-da3e0af83d62 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:23:32.641961 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e650706-235b-4336-9e3b-da3e0af83d62 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:23:32.653547 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 105/208] 10.222.0.33 () {56 vars in 1374 bytes} [Mon Jul 25 05:23:32 2022] GET /identity/v3/role_assignments?user.id=6fd6f234e1a54a4480852ab2c80c37c3&scope.project.id=4b840bd0540b4512b180fc68fd41cedc&role.id=be79956018b14baba7a021e2adf27084 => generated 608 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:23:33.400363 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a859b807-0c5c-4c02-9866-76ac93223bb6 None None] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:23:33.400925 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a859b807-0c5c-4c02-9866-76ac93223bb6 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:23:33.401006 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a859b807-0c5c-4c02-9866-76ac93223bb6 None None] PATH_INFO: `/` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:23:33.402665 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 104/209] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jul 25 05:23:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:23:33.404955 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb904547-d7e4-42d5-aa6a-a177a0806bd8 None None] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:23:33.406273 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb904547-d7e4-42d5-aa6a-a177a0806bd8 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:23:33.406771 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb904547-d7e4-42d5-aa6a-a177a0806bd8 None None] PATH_INFO: `/` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:23:33.407715 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac3417d-d2b8-4acf-bff3-b52119a3f3f3 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:23:33.408008 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac3417d-d2b8-4acf-bff3-b52119a3f3f3 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:23:33.408291 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac3417d-d2b8-4acf-bff3-b52119a3f3f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:23:33.409282 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 106/210] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jul 25 05:23:33 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:23:33.414735 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c423aefe-6b3b-4fa5-83d8-428493e4fd37 None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:23:33.416227 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c423aefe-6b3b-4fa5-83d8-428493e4fd37 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:23:33.416720 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c423aefe-6b3b-4fa5-83d8-428493e4fd37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:23:33.427572 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-0ac3417d-d2b8-4acf-bff3-b52119a3f3f3 None None] MFA Rules not processed for user `1eae540242d94c10b0ad33960a855f08`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:23:33.428767 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-0ac3417d-d2b8-4acf-bff3-b52119a3f3f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:23:33.441043 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-c423aefe-6b3b-4fa5-83d8-428493e4fd37 None None] MFA Rules not processed for user `1eae540242d94c10b0ad33960a855f08`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:23:33.442827 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-c423aefe-6b3b-4fa5-83d8-428493e4fd37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:23:33.490150 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:23:33.510301 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 105/211] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jul 25 05:23:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 05:23:33.528874 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 107/212] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jul 25 05:23:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 05:23:33.534870 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac3417d-d2b8-4acf-bff3-b52119a3f3f3 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:23:33.551282 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c423aefe-6b3b-4fa5-83d8-428493e4fd37 None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:23:33.644031 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac3417d-d2b8-4acf-bff3-b52119a3f3f3 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:23:33.651149 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c423aefe-6b3b-4fa5-83d8-428493e4fd37 None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:23:33.680902 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:23:33.682440 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd35e53-f934-4adf-9578-acc65fcdae02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:23:33.683497 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd35e53-f934-4adf-9578-acc65fcdae02 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:23:33.684063 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd35e53-f934-4adf-9578-acc65fcdae02 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:23:33.684630 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd35e53-f934-4adf-9578-acc65fcdae02 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:23:33.685706 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe41aff-7b5e-4e38-9bc5-aa319dc4d8c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:23:33.687146 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe41aff-7b5e-4e38-9bc5-aa319dc4d8c8 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:23:33.687622 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe41aff-7b5e-4e38-9bc5-aa319dc4d8c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:23:33.687879 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe41aff-7b5e-4e38-9bc5-aa319dc4d8c8 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:23:33.698408 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dd35e53-f934-4adf-9578-acc65fcdae02 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 05:23:33.698708 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebe41aff-7b5e-4e38-9bc5-aa319dc4d8c8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 05:23:33.706699 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application [None req-ebe41aff-7b5e-4e38-9bc5-aa319dc4d8c8 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 25 05:23:33.706699 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:23:33.706699 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:23:33.706699 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:23:33.706699 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:23:33.706699 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:23:33.706699 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:23:33.706699 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:23:33.706699 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:23:33.706699 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:23:33.706699 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:23:33.706699 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:23:33.706699 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 05:23:33.706699 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 05:23:33.706699 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 05:23:33.706699 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 05:23:33.707572 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 05:23:33.707572 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 05:23:33.707572 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 05:23:33.707572 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 05:23:33.707572 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:23:33.707572 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:23:33.707572 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:23:33.707572 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:23:33.707572 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:23:33.707572 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:23:33.707572 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:23:33.707572 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application with Lock( Jul 25 05:23:33.707572 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:23:33.707572 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:23:33.707572 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:23:33.707572 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:23:33.708351 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:23:33.708351 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:23:33.708351 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:23:33.708351 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:23:33.708351 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 05:23:33.708351 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 05:23:33.708351 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 05:23:33.708351 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 05:23:33.708351 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 05:23:33.708351 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 05:23:33.708351 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 25 05:23:33.708351 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Jul 25 05:23:33.708351 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 108/213] 10.222.0.33 () {56 vars in 1099 bytes} [Mon Jul 25 05:23:33 2022] GET /identity/v3/roles/member => generated 84 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 25 05:23:33.714674 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application [None req-7dd35e53-f934-4adf-9578-acc65fcdae02 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 05:23:33.714674 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:23:33.714674 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:23:33.714674 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:23:33.714674 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:23:33.714674 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:23:33.714674 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:23:33.714674 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:23:33.714674 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:23:33.714674 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:23:33.714674 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:23:33.714674 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:23:33.714674 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 05:23:33.714674 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 05:23:33.714674 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 05:23:33.714674 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 05:23:33.715621 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 05:23:33.715621 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 05:23:33.715621 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 05:23:33.715621 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 05:23:33.715621 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:23:33.715621 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:23:33.715621 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:23:33.715621 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:23:33.715621 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:23:33.715621 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:23:33.715621 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:23:33.715621 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application with Lock( Jul 25 05:23:33.715621 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:23:33.715621 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:23:33.715621 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:23:33.715621 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:23:33.718326 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:23:33.718326 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:23:33.718326 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:23:33.718326 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:23:33.718326 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 05:23:33.718326 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 05:23:33.718326 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 05:23:33.718326 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 05:23:33.718326 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 05:23:33.718326 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 05:23:33.718326 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 05:23:33.718326 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Jul 25 05:23:33.718958 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe41aff-7b5e-4e38-9bc5-aa319dc4d8c8 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:23:33.719029 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 106/214] 10.222.0.33 () {56 vars in 1096 bytes} [Mon Jul 25 05:23:33 2022] GET /identity/v3/roles/admin => generated 83 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:23:33.719284 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd35e53-f934-4adf-9578-acc65fcdae02 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:23:33.775165 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:23:33.833263 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd35e53-f934-4adf-9578-acc65fcdae02 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:23:33.836485 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe41aff-7b5e-4e38-9bc5-aa319dc4d8c8 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:23:33.868103 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86261296-f413-4bc4-be6d-11de93181fbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:23:33.868751 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86261296-f413-4bc4-be6d-11de93181fbd None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:23:33.869116 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86261296-f413-4bc4-be6d-11de93181fbd None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:23:33.869331 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86261296-f413-4bc4-be6d-11de93181fbd None admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:23:33.870143 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86261296-f413-4bc4-be6d-11de93181fbd None admin] RBAC: Adding query filter params (name=member) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:23:33.870455 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86261296-f413-4bc4-be6d-11de93181fbd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:23:33.871845 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86261296-f413-4bc4-be6d-11de93181fbd None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:23:33.873202 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c0153d-c2c9-4869-b961-6d85da7a7088 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:23:33.873895 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c0153d-c2c9-4869-b961-6d85da7a7088 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:23:33.874335 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c0153d-c2c9-4869-b961-6d85da7a7088 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:23:33.874617 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c0153d-c2c9-4869-b961-6d85da7a7088 None admin] PATH_INFO: `/v3/roles` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:23:33.875377 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2c0153d-c2c9-4869-b961-6d85da7a7088 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:23:33.875683 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2c0153d-c2c9-4869-b961-6d85da7a7088 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:23:33.877082 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2c0153d-c2c9-4869-b961-6d85da7a7088 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:23:33.881000 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 107/215] 10.222.0.33 () {56 vars in 1101 bytes} [Mon Jul 25 05:23:33 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:23:33.882126 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f870e31d-6164-48d3-a21c-8c8efb114918 None None] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:23:33.882414 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f870e31d-6164-48d3-a21c-8c8efb114918 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:23:33.882688 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f870e31d-6164-48d3-a21c-8c8efb114918 None None] PATH_INFO: `/` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:23:33.884053 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 108/216] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jul 25 05:23:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:23:33.886362 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f870e31d-6164-48d3-a21c-8c8efb114918 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:23:33.892158 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 109/217] 10.222.0.33 () {56 vars in 1099 bytes} [Mon Jul 25 05:23:33 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:23:33.893704 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04785d6-23ae-44bc-aab5-b2f59ecd58e1 None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:23:33.894207 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04785d6-23ae-44bc-aab5-b2f59ecd58e1 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:23:33.894694 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04785d6-23ae-44bc-aab5-b2f59ecd58e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:23:33.914309 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-c04785d6-23ae-44bc-aab5-b2f59ecd58e1 None None] MFA Rules not processed for user `1eae540242d94c10b0ad33960a855f08`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:23:33.915433 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-c04785d6-23ae-44bc-aab5-b2f59ecd58e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:23:33.933267 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:23:33.990184 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 110/218] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jul 25 05:23:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 05:23:33.996569 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f870e31d-6164-48d3-a21c-8c8efb114918 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:23:33.997730 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c04785d6-23ae-44bc-aab5-b2f59ecd58e1 None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:23:34.006558 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:23:34.036095 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76512155-1e4c-4630-879b-87c98f817be9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:23:34.037350 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76512155-1e4c-4630-879b-87c98f817be9 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:23:34.037796 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76512155-1e4c-4630-879b-87c98f817be9 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:23:34.038071 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76512155-1e4c-4630-879b-87c98f817be9 None admin] PATH_INFO: `/v3/users/0c23382cdaf5487993b90e76e5c60721` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:23:34.052530 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76512155-1e4c-4630-879b-87c98f817be9 None admin] RBAC: Authorizing `identity:get_user(user_id=0c23382cdaf5487993b90e76e5c60721)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:23:34.054046 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76512155-1e4c-4630-879b-87c98f817be9 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:23:34.055770 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 109/219] 10.222.0.33 () {56 vars in 1177 bytes} [Mon Jul 25 05:23:33 2022] GET /identity/v3/users/0c23382cdaf5487993b90e76e5c60721 => generated 297 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 05:23:34.055770 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76512155-1e4c-4630-879b-87c98f817be9 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:23:34.111500 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c04785d6-23ae-44bc-aab5-b2f59ecd58e1 None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:23:34.146587 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b9cc4a-a75f-40ab-8d2a-4062a3cde312 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:23:34.147469 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b9cc4a-a75f-40ab-8d2a-4062a3cde312 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:23:34.147759 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b9cc4a-a75f-40ab-8d2a-4062a3cde312 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:23:34.148036 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b9cc4a-a75f-40ab-8d2a-4062a3cde312 None admin] PATH_INFO: `/v3/users/1c6dda23da0644c2b35c2b1ffc72a8f1` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:23:34.149273 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99b9cc4a-a75f-40ab-8d2a-4062a3cde312 None admin] RBAC: Authorizing `identity:get_user(user_id=1c6dda23da0644c2b35c2b1ffc72a8f1)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:23:34.150727 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99b9cc4a-a75f-40ab-8d2a-4062a3cde312 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:23:34.152563 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 111/220] 10.222.0.33 () {56 vars in 1177 bytes} [Mon Jul 25 05:23:33 2022] GET /identity/v3/users/1c6dda23da0644c2b35c2b1ffc72a8f1 => generated 283 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:23:34.153171 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b9cc4a-a75f-40ab-8d2a-4062a3cde312 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:23:34.169823 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76512155-1e4c-4630-879b-87c98f817be9 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:23:34.175105 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:23:34.204791 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a214d13-c13e-4eb8-b34a-9b3b7ce8411a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:23:34.205601 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a214d13-c13e-4eb8-b34a-9b3b7ce8411a None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:23:34.205882 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a214d13-c13e-4eb8-b34a-9b3b7ce8411a None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:23:34.206162 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a214d13-c13e-4eb8-b34a-9b3b7ce8411a None admin] PATH_INFO: `/v3/domains/default` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:23:34.207330 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a214d13-c13e-4eb8-b34a-9b3b7ce8411a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:23:34.208761 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a214d13-c13e-4eb8-b34a-9b3b7ce8411a None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:23:34.210407 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 110/221] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Jul 25 05:23:34 2022] GET /identity/v3/domains/default => generated 199 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:23:34.211162 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a214d13-c13e-4eb8-b34a-9b3b7ce8411a None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:23:34.241337 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:23:34.259910 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b9cc4a-a75f-40ab-8d2a-4062a3cde312 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:23:34.293632 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28364d0f-77e5-43e4-a1c9-f8262bd9ebc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:23:34.294281 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28364d0f-77e5-43e4-a1c9-f8262bd9ebc9 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:23:34.294571 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28364d0f-77e5-43e4-a1c9-f8262bd9ebc9 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:23:34.294849 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28364d0f-77e5-43e4-a1c9-f8262bd9ebc9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:23:34.308104 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a214d13-c13e-4eb8-b34a-9b3b7ce8411a None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:23:34.341644 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b38a60d-7f30-4d56-b9fc-1a79505fa716 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:23:34.342494 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b38a60d-7f30-4d56-b9fc-1a79505fa716 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:23:34.343383 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b38a60d-7f30-4d56-b9fc-1a79505fa716 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:23:34.343659 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b38a60d-7f30-4d56-b9fc-1a79505fa716 None admin] PATH_INFO: `/v3/projects/ec1ed227a23f453cab64e15036af0c60` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:23:34.344895 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b38a60d-7f30-4d56-b9fc-1a79505fa716 None admin] RBAC: Authorizing `identity:get_project(project_id=ec1ed227a23f453cab64e15036af0c60)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:23:34.346340 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b38a60d-7f30-4d56-b9fc-1a79505fa716 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:23:34.348032 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 111/222] 10.222.0.33 () {56 vars in 1186 bytes} [Mon Jul 25 05:23:34 2022] GET /identity/v3/projects/ec1ed227a23f453cab64e15036af0c60 => generated 302 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:23:34.349217 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e84e2c-8912-493e-84af-d4f8a52c55b2 None None] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:23:34.349493 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e84e2c-8912-493e-84af-d4f8a52c55b2 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:23:34.349768 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e84e2c-8912-493e-84af-d4f8a52c55b2 None None] PATH_INFO: `/` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:23:34.351183 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 112/223] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jul 25 05:23:34 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:23:34.351669 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e84e2c-8912-493e-84af-d4f8a52c55b2 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:23:34.366143 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28364d0f-77e5-43e4-a1c9-f8262bd9ebc9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:23:34.367652 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28364d0f-77e5-43e4-a1c9-f8262bd9ebc9 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:23:34.388221 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:23:34.449597 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:23:34.453973 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e84e2c-8912-493e-84af-d4f8a52c55b2 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:23:34.470107 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 112/224] 10.222.0.33 () {58 vars in 1282 bytes} [Mon Jul 25 05:23:34 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 25 05:23:34.470107 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28364d0f-77e5-43e4-a1c9-f8262bd9ebc9 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:23:34.487237 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a050fafd-f0e6-453e-ae48-7ac1502dd505 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:23:34.487916 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a050fafd-f0e6-453e-ae48-7ac1502dd505 None admin] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:23:34.488204 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a050fafd-f0e6-453e-ae48-7ac1502dd505 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:23:34.488483 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a050fafd-f0e6-453e-ae48-7ac1502dd505 None admin] PATH_INFO: `/v3/users` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:23:34.489237 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a050fafd-f0e6-453e-ae48-7ac1502dd505 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:23:34.490682 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a050fafd-f0e6-453e-ae48-7ac1502dd505 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:23:34.511364 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 113/225] 10.222.0.33 () {60 vars in 1132 bytes} [Mon Jul 25 05:23:34 2022] POST /identity/v3/users => generated 297 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 05:23:34.512665 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84323c4-3aad-499c-a117-ba2f1f6ff110 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:23:34.512984 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84323c4-3aad-499c-a117-ba2f1f6ff110 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:23:34.513260 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84323c4-3aad-499c-a117-ba2f1f6ff110 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:23:34.547195 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-f84323c4-3aad-499c-a117-ba2f1f6ff110 None None] MFA Rules not processed for user `1eae540242d94c10b0ad33960a855f08`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:23:34.548417 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-f84323c4-3aad-499c-a117-ba2f1f6ff110 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:23:34.575944 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28364d0f-77e5-43e4-a1c9-f8262bd9ebc9 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:23:34.603333 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:23:34.619840 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0412cb5-6f70-4199-a104-de1252d63546 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:23:34.621084 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0412cb5-6f70-4199-a104-de1252d63546 None admin] REQUEST_METHOD: `PUT` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:23:34.621524 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0412cb5-6f70-4199-a104-de1252d63546 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:23:34.621803 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0412cb5-6f70-4199-a104-de1252d63546 None admin] PATH_INFO: `/v3/projects/ec1ed227a23f453cab64e15036af0c60/users/1c6dda23da0644c2b35c2b1ffc72a8f1/roles/39e5c666285a42fe8fc2c3afca6260d1` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:23:34.623918 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0412cb5-6f70-4199-a104-de1252d63546 None admin] RBAC: Authorizing `identity:create_grant(project_id=ec1ed227a23f453cab64e15036af0c60, user_id=1c6dda23da0644c2b35c2b1ffc72a8f1, role_id=39e5c666285a42fe8fc2c3afca6260d1)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:23:34.625589 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0412cb5-6f70-4199-a104-de1252d63546 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:23:34.632899 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 114/226] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jul 25 05:23:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 05:23:34.632899 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f84323c4-3aad-499c-a117-ba2f1f6ff110 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:23:34.652193 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 113/227] 10.222.0.33 () {58 vars in 1439 bytes} [Mon Jul 25 05:23:34 2022] PUT /identity/v3/projects/ec1ed227a23f453cab64e15036af0c60/users/1c6dda23da0644c2b35c2b1ffc72a8f1/roles/39e5c666285a42fe8fc2c3afca6260d1 => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 05:23:34.653117 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0412cb5-6f70-4199-a104-de1252d63546 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:23:34.760574 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:23:34.785130 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f84323c4-3aad-499c-a117-ba2f1f6ff110 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:23:34.791719 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0412cb5-6f70-4199-a104-de1252d63546 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:23:34.836449 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c69fe91-d68f-4a48-a18f-dbb0244654de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:23:34.837371 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c69fe91-d68f-4a48-a18f-dbb0244654de None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:23:34.837655 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c69fe91-d68f-4a48-a18f-dbb0244654de None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:23:34.837934 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c69fe91-d68f-4a48-a18f-dbb0244654de None admin] PATH_INFO: `/v3/projects/ec1ed227a23f453cab64e15036af0c60` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:23:34.839847 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c69fe91-d68f-4a48-a18f-dbb0244654de None admin] RBAC: Authorizing `identity:get_project(project_id=ec1ed227a23f453cab64e15036af0c60)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:23:34.841340 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c69fe91-d68f-4a48-a18f-dbb0244654de None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:23:34.841460 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c54b1e-f02e-4338-b49e-3ce2b339be2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:23:34.842215 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c54b1e-f02e-4338-b49e-3ce2b339be2a None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:23:34.842527 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c54b1e-f02e-4338-b49e-3ce2b339be2a None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:23:34.842982 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c54b1e-f02e-4338-b49e-3ce2b339be2a None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:23:34.843600 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 115/228] 10.222.0.33 () {56 vars in 1186 bytes} [Mon Jul 25 05:23:34 2022] GET /identity/v3/projects/ec1ed227a23f453cab64e15036af0c60 => generated 302 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:23:34.850864 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c69fe91-d68f-4a48-a18f-dbb0244654de None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:23:34.857811 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4c54b1e-f02e-4338-b49e-3ce2b339be2a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 05:23:34.866718 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application [None req-c4c54b1e-f02e-4338-b49e-3ce2b339be2a None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 25 05:23:34.866718 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:23:34.866718 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:23:34.866718 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:23:34.866718 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:23:34.866718 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:23:34.866718 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:23:34.866718 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:23:34.866718 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:23:34.866718 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:23:34.866718 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:23:34.866718 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:23:34.866718 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 05:23:34.866718 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 05:23:34.866718 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 05:23:34.866718 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 05:23:34.867648 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 05:23:34.867648 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 05:23:34.867648 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 05:23:34.867648 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 05:23:34.867648 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:23:34.867648 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:23:34.867648 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:23:34.867648 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:23:34.867648 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:23:34.867648 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:23:34.867648 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:23:34.867648 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application with Lock( Jul 25 05:23:34.867648 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:23:34.867648 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:23:34.867648 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:23:34.867648 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:23:34.868636 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:23:34.868636 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:23:34.868636 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:23:34.868636 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:23:34.868636 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 05:23:34.868636 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 05:23:34.868636 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 05:23:34.868636 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 05:23:34.868636 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 05:23:34.868636 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 05:23:34.868636 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 25 05:23:34.868636 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Jul 25 05:23:34.868636 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 114/229] 10.222.0.33 () {56 vars in 1114 bytes} [Mon Jul 25 05:23:34 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:23:34.873642 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c54b1e-f02e-4338-b49e-3ce2b339be2a None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:23:34.925098 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:23:34.978607 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c69fe91-d68f-4a48-a18f-dbb0244654de None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:23:35.001909 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c54b1e-f02e-4338-b49e-3ce2b339be2a None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:23:35.017318 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e3908db-b48c-4264-95bc-087b954fc7e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:23:35.018068 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3908db-b48c-4264-95bc-087b954fc7e7 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:23:35.018424 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3908db-b48c-4264-95bc-087b954fc7e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:23:35.018770 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3908db-b48c-4264-95bc-087b954fc7e7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:23:35.019746 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e3908db-b48c-4264-95bc-087b954fc7e7 None admin] RBAC: Adding query filter params (role.id=5074e74e35f54d4b9bb415c34999a82a, scope.project.id=ec1ed227a23f453cab64e15036af0c60, user.id=0c23382cdaf5487993b90e76e5c60721) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:23:35.020145 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e3908db-b48c-4264-95bc-087b954fc7e7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:23:35.021698 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e3908db-b48c-4264-95bc-087b954fc7e7 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:23:35.030053 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 116/230] 10.222.0.33 () {56 vars in 1374 bytes} [Mon Jul 25 05:23:34 2022] GET /identity/v3/role_assignments?user.id=0c23382cdaf5487993b90e76e5c60721&scope.project.id=ec1ed227a23f453cab64e15036af0c60&role.id=5074e74e35f54d4b9bb415c34999a82a => generated 259 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 05:23:35.051509 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec9ade8-19f2-4b89-9755-993bb9d6b29d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:23:35.052203 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec9ade8-19f2-4b89-9755-993bb9d6b29d None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:23:35.052489 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec9ade8-19f2-4b89-9755-993bb9d6b29d None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:23:35.052771 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec9ade8-19f2-4b89-9755-993bb9d6b29d None admin] PATH_INFO: `/v3/roles` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:23:35.053556 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ec9ade8-19f2-4b89-9755-993bb9d6b29d None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:23:35.053868 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ec9ade8-19f2-4b89-9755-993bb9d6b29d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:23:35.055262 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ec9ade8-19f2-4b89-9755-993bb9d6b29d None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:23:35.069312 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 115/231] 10.222.0.33 () {56 vars in 1111 bytes} [Mon Jul 25 05:23:34 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 05:23:35.074735 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e3908db-b48c-4264-95bc-087b954fc7e7 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:23:35.118693 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:23:35.233790 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e3908db-b48c-4264-95bc-087b954fc7e7 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:23:35.278347 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f942da-9bec-49a2-ade0-a8c572cbfcf1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:23:35.279154 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f942da-9bec-49a2-ade0-a8c572cbfcf1 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:23:35.279442 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f942da-9bec-49a2-ade0-a8c572cbfcf1 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:23:35.279724 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f942da-9bec-49a2-ade0-a8c572cbfcf1 None admin] PATH_INFO: `/v3/users/1c6dda23da0644c2b35c2b1ffc72a8f1` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:23:35.281705 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81f942da-9bec-49a2-ade0-a8c572cbfcf1 None admin] RBAC: Authorizing `identity:get_user(user_id=1c6dda23da0644c2b35c2b1ffc72a8f1)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:23:35.283161 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81f942da-9bec-49a2-ade0-a8c572cbfcf1 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:23:35.285267 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 117/232] 10.222.0.33 () {56 vars in 1177 bytes} [Mon Jul 25 05:23:35 2022] GET /identity/v3/users/1c6dda23da0644c2b35c2b1ffc72a8f1 => generated 283 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:23:35.290414 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec9ade8-19f2-4b89-9755-993bb9d6b29d None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:23:35.418095 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec9ade8-19f2-4b89-9755-993bb9d6b29d None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:23:35.419699 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:23:35.464731 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5430ca0a-e306-4b21-8ae6-05083c14eeab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:23:35.465617 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5430ca0a-e306-4b21-8ae6-05083c14eeab None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:23:35.465902 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5430ca0a-e306-4b21-8ae6-05083c14eeab None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:23:35.466178 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5430ca0a-e306-4b21-8ae6-05083c14eeab None admin] PATH_INFO: `/v3/projects/ec1ed227a23f453cab64e15036af0c60` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:23:35.473559 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5430ca0a-e306-4b21-8ae6-05083c14eeab None admin] RBAC: Authorizing `identity:get_project(project_id=ec1ed227a23f453cab64e15036af0c60)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:23:35.475046 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5430ca0a-e306-4b21-8ae6-05083c14eeab None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:23:35.477017 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 116/233] 10.222.0.33 () {56 vars in 1186 bytes} [Mon Jul 25 05:23:35 2022] GET /identity/v3/projects/ec1ed227a23f453cab64e15036af0c60 => generated 302 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:23:35.483021 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f942da-9bec-49a2-ade0-a8c572cbfcf1 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:23:35.611146 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:23:35.630577 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f942da-9bec-49a2-ade0-a8c572cbfcf1 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:23:35.680518 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be22d7cc-c014-448a-a185-3f40c587dd1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:23:35.681496 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be22d7cc-c014-448a-a185-3f40c587dd1a None admin] REQUEST_METHOD: `PUT` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:23:35.681779 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be22d7cc-c014-448a-a185-3f40c587dd1a None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:23:35.682048 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be22d7cc-c014-448a-a185-3f40c587dd1a None admin] PATH_INFO: `/v3/projects/ec1ed227a23f453cab64e15036af0c60/users/1c6dda23da0644c2b35c2b1ffc72a8f1/roles/02aba1f7cbdf4d6595e18af7fb5727e0` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:23:35.684468 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be22d7cc-c014-448a-a185-3f40c587dd1a None admin] RBAC: Authorizing `identity:create_grant(project_id=ec1ed227a23f453cab64e15036af0c60, user_id=1c6dda23da0644c2b35c2b1ffc72a8f1, role_id=02aba1f7cbdf4d6595e18af7fb5727e0)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:23:35.685969 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be22d7cc-c014-448a-a185-3f40c587dd1a None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:23:35.716592 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 118/234] 10.222.0.33 () {58 vars in 1439 bytes} [Mon Jul 25 05:23:35 2022] PUT /identity/v3/projects/ec1ed227a23f453cab64e15036af0c60/users/1c6dda23da0644c2b35c2b1ffc72a8f1/roles/02aba1f7cbdf4d6595e18af7fb5727e0 => generated 0 bytes in 234 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 05:23:37.068282 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3aef88b-9d92-4d03-b937-a0f4830ca9c2 None None] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:23:37.070964 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3aef88b-9d92-4d03-b937-a0f4830ca9c2 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:23:37.070964 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3aef88b-9d92-4d03-b937-a0f4830ca9c2 None None] PATH_INFO: `/` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:23:37.070964 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 117/235] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jul 25 05:23:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:23:37.076370 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541b1317-8f5a-4ccb-ab3c-bc328e64bf9f None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:23:37.077342 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541b1317-8f5a-4ccb-ab3c-bc328e64bf9f None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:23:37.077877 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541b1317-8f5a-4ccb-ab3c-bc328e64bf9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:23:37.095222 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-541b1317-8f5a-4ccb-ab3c-bc328e64bf9f None None] MFA Rules not processed for user `1eae540242d94c10b0ad33960a855f08`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:23:37.096757 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-541b1317-8f5a-4ccb-ab3c-bc328e64bf9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:23:37.179457 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 119/236] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jul 25 05:23:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 05:23:37.215358 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3aef88b-9d92-4d03-b937-a0f4830ca9c2 None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:23:37.239327 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d27c35-bbc1-4e34-869d-4864828009a0 None None] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:23:37.239937 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d27c35-bbc1-4e34-869d-4864828009a0 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:23:37.240426 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d27c35-bbc1-4e34-869d-4864828009a0 None None] PATH_INFO: `/` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:23:37.242041 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 120/237] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jul 25 05:23:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 25 05:23:37.250785 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232f09de-7abc-4d1b-87ca-ee952dc86950 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:23:37.251355 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232f09de-7abc-4d1b-87ca-ee952dc86950 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:23:37.251829 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232f09de-7abc-4d1b-87ca-ee952dc86950 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:23:37.270094 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-232f09de-7abc-4d1b-87ca-ee952dc86950 None None] MFA Rules not processed for user `1eae540242d94c10b0ad33960a855f08`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:23:37.271671 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-232f09de-7abc-4d1b-87ca-ee952dc86950 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:23:37.367112 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 121/238] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jul 25 05:23:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 25 05:23:37.390464 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3aef88b-9d92-4d03-b937-a0f4830ca9c2 None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:23:37.415712 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-232f09de-7abc-4d1b-87ca-ee952dc86950 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:23:37.423873 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:23:37.445525 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9bdb373-1912-48d3-93d5-f8c1e2569e46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:23:37.446340 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9bdb373-1912-48d3-93d5-f8c1e2569e46 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:23:37.446628 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9bdb373-1912-48d3-93d5-f8c1e2569e46 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:23:37.446907 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9bdb373-1912-48d3-93d5-f8c1e2569e46 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:23:37.463739 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9bdb373-1912-48d3-93d5-f8c1e2569e46 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 05:23:37.471489 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application [None req-a9bdb373-1912-48d3-93d5-f8c1e2569e46 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 05:23:37.471489 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:23:37.471489 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:23:37.471489 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:23:37.471489 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:23:37.471489 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:23:37.471489 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:23:37.471489 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:23:37.471489 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:23:37.471489 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:23:37.471489 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:23:37.471489 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:23:37.471489 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 05:23:37.471489 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 05:23:37.471489 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 05:23:37.471489 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 05:23:37.472357 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 05:23:37.472357 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 05:23:37.472357 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 05:23:37.472357 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 05:23:37.472357 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:23:37.472357 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:23:37.472357 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:23:37.472357 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:23:37.472357 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:23:37.472357 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:23:37.472357 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:23:37.472357 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application with Lock( Jul 25 05:23:37.472357 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:23:37.472357 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:23:37.472357 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:23:37.472357 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:23:37.473180 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:23:37.473180 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:23:37.473180 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:23:37.473180 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:23:37.473180 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 05:23:37.473180 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 05:23:37.473180 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 05:23:37.473180 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 05:23:37.473180 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 05:23:37.473180 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 05:23:37.473180 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 05:23:37.473180 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Jul 25 05:23:37.474272 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 118/239] 10.222.0.33 () {56 vars in 1096 bytes} [Mon Jul 25 05:23:37 2022] GET /identity/v3/roles/admin => generated 83 bytes in 259 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:23:37.484747 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9bdb373-1912-48d3-93d5-f8c1e2569e46 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:23:37.524908 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:23:37.565386 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-232f09de-7abc-4d1b-87ca-ee952dc86950 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:23:37.601691 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4220766b-c6e1-4d85-bff4-933df5bb2ff5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:23:37.602763 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4220766b-c6e1-4d85-bff4-933df5bb2ff5 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:23:37.603191 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4220766b-c6e1-4d85-bff4-933df5bb2ff5 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:23:37.603611 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4220766b-c6e1-4d85-bff4-933df5bb2ff5 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:23:37.616867 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4220766b-c6e1-4d85-bff4-933df5bb2ff5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 05:23:37.619079 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9bdb373-1912-48d3-93d5-f8c1e2569e46 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:23:37.625897 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application [None req-4220766b-c6e1-4d85-bff4-933df5bb2ff5 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 25 05:23:37.625897 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:23:37.625897 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:23:37.625897 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:23:37.625897 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:23:37.625897 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:23:37.625897 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:23:37.625897 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:23:37.625897 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:23:37.625897 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:23:37.625897 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:23:37.625897 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:23:37.625897 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 05:23:37.625897 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 05:23:37.625897 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 05:23:37.625897 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 05:23:37.626790 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 05:23:37.626790 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 05:23:37.626790 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 05:23:37.626790 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 05:23:37.626790 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:23:37.626790 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:23:37.626790 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:23:37.626790 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:23:37.626790 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:23:37.626790 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:23:37.626790 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:23:37.626790 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application with Lock( Jul 25 05:23:37.626790 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:23:37.626790 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:23:37.626790 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:23:37.626790 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:23:37.627589 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:23:37.627589 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:23:37.627589 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:23:37.627589 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:23:37.627589 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 05:23:37.627589 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 05:23:37.627589 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 05:23:37.627589 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 05:23:37.627589 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 05:23:37.627589 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 05:23:37.627589 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 25 05:23:37.627589 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Jul 25 05:23:37.630004 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 122/240] 10.222.0.33 () {56 vars in 1099 bytes} [Mon Jul 25 05:23:37 2022] GET /identity/v3/roles/reader => generated 84 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 25 05:23:37.631412 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21380893-cf82-4a30-bb15-45797836c0b7 None None] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:23:37.631892 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21380893-cf82-4a30-bb15-45797836c0b7 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:23:37.632357 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21380893-cf82-4a30-bb15-45797836c0b7 None None] PATH_INFO: `/` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:23:37.633921 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 123/241] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jul 25 05:23:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:23:37.635527 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21380893-cf82-4a30-bb15-45797836c0b7 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:23:37.667526 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-068d4bfa-e89d-48a2-bd22-3ff096578262 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:23:37.668339 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068d4bfa-e89d-48a2-bd22-3ff096578262 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:23:37.668746 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068d4bfa-e89d-48a2-bd22-3ff096578262 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:23:37.669076 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068d4bfa-e89d-48a2-bd22-3ff096578262 None admin] PATH_INFO: `/v3/roles` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:23:37.669817 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-068d4bfa-e89d-48a2-bd22-3ff096578262 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:23:37.670234 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-068d4bfa-e89d-48a2-bd22-3ff096578262 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:23:37.672355 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-068d4bfa-e89d-48a2-bd22-3ff096578262 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:23:37.683450 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 119/242] 10.222.0.33 () {56 vars in 1099 bytes} [Mon Jul 25 05:23:37 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:23:37.685260 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87340ebb-4fe3-4b83-8c8e-8de3cca73d35 None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:23:37.685548 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87340ebb-4fe3-4b83-8c8e-8de3cca73d35 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:23:37.685826 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87340ebb-4fe3-4b83-8c8e-8de3cca73d35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:23:37.706424 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-87340ebb-4fe3-4b83-8c8e-8de3cca73d35 None None] MFA Rules not processed for user `1eae540242d94c10b0ad33960a855f08`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:23:37.707619 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-87340ebb-4fe3-4b83-8c8e-8de3cca73d35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:23:37.711228 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:23:37.751177 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21380893-cf82-4a30-bb15-45797836c0b7 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:23:37.779064 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 120/243] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jul 25 05:23:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 05:23:37.782055 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87340ebb-4fe3-4b83-8c8e-8de3cca73d35 None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:23:37.786110 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a18c3165-c71d-4f6d-bbeb-d7d1ef3e276b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:23:37.786788 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18c3165-c71d-4f6d-bbeb-d7d1ef3e276b None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:23:37.787075 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18c3165-c71d-4f6d-bbeb-d7d1ef3e276b None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:23:37.787351 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18c3165-c71d-4f6d-bbeb-d7d1ef3e276b None admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:23:37.788110 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a18c3165-c71d-4f6d-bbeb-d7d1ef3e276b None admin] RBAC: Adding query filter params (name=reader) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:23:37.788418 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a18c3165-c71d-4f6d-bbeb-d7d1ef3e276b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:23:37.789829 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a18c3165-c71d-4f6d-bbeb-d7d1ef3e276b None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:23:37.804055 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 124/244] 10.222.0.33 () {56 vars in 1101 bytes} [Mon Jul 25 05:23:37 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 05:23:37.805553 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517651e6-c052-4c58-b4c7-fa98f078edfb None None] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:23:37.805849 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517651e6-c052-4c58-b4c7-fa98f078edfb None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:23:37.806131 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517651e6-c052-4c58-b4c7-fa98f078edfb None None] PATH_INFO: `/` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:23:37.811488 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 125/245] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jul 25 05:23:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:23:37.811488 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-517651e6-c052-4c58-b4c7-fa98f078edfb None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:23:37.823944 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:23:37.907320 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87340ebb-4fe3-4b83-8c8e-8de3cca73d35 None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:23:37.921386 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-517651e6-c052-4c58-b4c7-fa98f078edfb None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:23:37.941936 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a4c0f07-c527-41e2-88d2-d080e8151e46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:23:37.942814 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4c0f07-c527-41e2-88d2-d080e8151e46 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:23:37.943432 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4c0f07-c527-41e2-88d2-d080e8151e46 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:23:37.943432 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4c0f07-c527-41e2-88d2-d080e8151e46 None admin] PATH_INFO: `/v3/users/1c6dda23da0644c2b35c2b1ffc72a8f1` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:23:37.944567 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a4c0f07-c527-41e2-88d2-d080e8151e46 None admin] RBAC: Authorizing `identity:get_user(user_id=1c6dda23da0644c2b35c2b1ffc72a8f1)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:23:37.946057 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a4c0f07-c527-41e2-88d2-d080e8151e46 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:23:37.952514 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 121/246] 10.222.0.33 () {56 vars in 1177 bytes} [Mon Jul 25 05:23:37 2022] GET /identity/v3/users/1c6dda23da0644c2b35c2b1ffc72a8f1 => generated 283 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:23:37.952514 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a4c0f07-c527-41e2-88d2-d080e8151e46 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:23:37.955980 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:23:37.964235 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-907f3b0e-e67a-41e7-9d32-d5bd38727b9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:23:37.965158 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907f3b0e-e67a-41e7-9d32-d5bd38727b9d None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:23:37.965450 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907f3b0e-e67a-41e7-9d32-d5bd38727b9d None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:23:37.965719 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907f3b0e-e67a-41e7-9d32-d5bd38727b9d None admin] PATH_INFO: `/v3/users/e03df72adbd34b80951f6d287621b9b9` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:23:37.980419 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-907f3b0e-e67a-41e7-9d32-d5bd38727b9d None admin] RBAC: Authorizing `identity:get_user(user_id=e03df72adbd34b80951f6d287621b9b9)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:23:37.981913 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-907f3b0e-e67a-41e7-9d32-d5bd38727b9d None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:23:37.984000 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 126/247] 10.222.0.33 () {56 vars in 1177 bytes} [Mon Jul 25 05:23:37 2022] GET /identity/v3/users/e03df72adbd34b80951f6d287621b9b9 => generated 297 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:23:37.989391 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e93b572-8fae-4303-9e62-b45c1ee43ce8 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:23:37.989684 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e93b572-8fae-4303-9e62-b45c1ee43ce8 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:23:37.989971 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e93b572-8fae-4303-9e62-b45c1ee43ce8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:23:38.006772 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-4e93b572-8fae-4303-9e62-b45c1ee43ce8 None None] MFA Rules not processed for user `1eae540242d94c10b0ad33960a855f08`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:23:38.007960 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-4e93b572-8fae-4303-9e62-b45c1ee43ce8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:23:38.050582 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:23:38.062475 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a4c0f07-c527-41e2-88d2-d080e8151e46 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:23:38.080502 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 127/248] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jul 25 05:23:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 05:23:38.081318 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e93b572-8fae-4303-9e62-b45c1ee43ce8 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:23:38.103687 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd49585-8433-4186-bc4b-e1e80f5002de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:23:38.104595 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd49585-8433-4186-bc4b-e1e80f5002de None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:23:38.104916 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd49585-8433-4186-bc4b-e1e80f5002de None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:23:38.105199 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd49585-8433-4186-bc4b-e1e80f5002de None admin] PATH_INFO: `/v3/domains/default` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:23:38.106395 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd49585-8433-4186-bc4b-e1e80f5002de None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:23:38.108028 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd49585-8433-4186-bc4b-e1e80f5002de None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:23:38.110458 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 122/249] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Jul 25 05:23:37 2022] GET /identity/v3/domains/default => generated 199 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:23:38.112892 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd49585-8433-4186-bc4b-e1e80f5002de None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:23:38.187495 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e93b572-8fae-4303-9e62-b45c1ee43ce8 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:23:38.204688 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:23:38.223710 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd49585-8433-4186-bc4b-e1e80f5002de None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:23:38.223992 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c177912-c11b-42d4-8aa0-f332b600f546 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:23:38.224454 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c177912-c11b-42d4-8aa0-f332b600f546 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:23:38.224750 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c177912-c11b-42d4-8aa0-f332b600f546 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:23:38.225060 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c177912-c11b-42d4-8aa0-f332b600f546 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:23:38.261927 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf98ea8-b964-40cc-a8f2-10ac1370954f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:23:38.262589 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf98ea8-b964-40cc-a8f2-10ac1370954f None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:23:38.262913 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf98ea8-b964-40cc-a8f2-10ac1370954f None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:23:38.263194 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf98ea8-b964-40cc-a8f2-10ac1370954f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:23:38.290916 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:23:38.295222 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c177912-c11b-42d4-8aa0-f332b600f546 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:23:38.297471 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c177912-c11b-42d4-8aa0-f332b600f546 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:23:38.332407 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddf98ea8-b964-40cc-a8f2-10ac1370954f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:23:38.333913 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddf98ea8-b964-40cc-a8f2-10ac1370954f None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:23:38.395371 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 128/250] 10.222.0.33 () {58 vars in 1282 bytes} [Mon Jul 25 05:23:38 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 25 05:23:38.396409 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c177912-c11b-42d4-8aa0-f332b600f546 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:23:38.416244 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:23:38.431915 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 123/251] 10.222.0.33 () {58 vars in 1282 bytes} [Mon Jul 25 05:23:38 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 25 05:23:38.432582 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf98ea8-b964-40cc-a8f2-10ac1370954f None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:23:38.489105 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:23:38.501653 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c177912-c11b-42d4-8aa0-f332b600f546 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:23:38.530301 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf98ea8-b964-40cc-a8f2-10ac1370954f None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:23:38.534107 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-873fcd93-ec47-4226-a802-c2a107ddf29c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:23:38.534893 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873fcd93-ec47-4226-a802-c2a107ddf29c None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:23:38.535176 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873fcd93-ec47-4226-a802-c2a107ddf29c None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:23:38.535453 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873fcd93-ec47-4226-a802-c2a107ddf29c None admin] PATH_INFO: `/v3/roles/member` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:23:38.546910 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-873fcd93-ec47-4226-a802-c2a107ddf29c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 05:23:38.554609 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application [None req-873fcd93-ec47-4226-a802-c2a107ddf29c None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 25 05:23:38.554609 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:23:38.554609 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:23:38.554609 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:23:38.554609 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:23:38.554609 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:23:38.554609 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:23:38.554609 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:23:38.554609 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:23:38.554609 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:23:38.554609 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:23:38.554609 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:23:38.554609 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 05:23:38.554609 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 05:23:38.554609 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 05:23:38.554609 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 05:23:38.555476 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 05:23:38.555476 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 05:23:38.555476 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 05:23:38.555476 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 05:23:38.555476 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:23:38.555476 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:23:38.555476 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:23:38.555476 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:23:38.555476 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:23:38.555476 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:23:38.555476 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:23:38.555476 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application with Lock( Jul 25 05:23:38.555476 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:23:38.555476 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:23:38.555476 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:23:38.555476 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:23:38.556746 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:23:38.556746 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:23:38.556746 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:23:38.556746 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:23:38.556746 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 05:23:38.556746 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 05:23:38.556746 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 05:23:38.556746 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 05:23:38.556746 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 05:23:38.556746 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 05:23:38.556746 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 25 05:23:38.556746 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Jul 25 05:23:38.556746 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 129/252] 10.222.0.33 () {56 vars in 1099 bytes} [Mon Jul 25 05:23:38 2022] GET /identity/v3/roles/member => generated 84 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:23:38.557721 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c88082-a432-4568-bf6c-497960e4c9b0 None None] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:23:38.557721 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c88082-a432-4568-bf6c-497960e4c9b0 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:23:38.557721 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c88082-a432-4568-bf6c-497960e4c9b0 None None] PATH_INFO: `/` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:23:38.559053 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 130/253] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jul 25 05:23:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:23:38.559753 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c88082-a432-4568-bf6c-497960e4c9b0 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:23:38.571343 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9846c945-0821-41b6-91f1-ba7b9956c183 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:23:38.572016 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9846c945-0821-41b6-91f1-ba7b9956c183 None admin] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:23:38.572293 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9846c945-0821-41b6-91f1-ba7b9956c183 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:23:38.572575 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9846c945-0821-41b6-91f1-ba7b9956c183 None admin] PATH_INFO: `/v3/users` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:23:38.573326 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9846c945-0821-41b6-91f1-ba7b9956c183 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:23:38.574839 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9846c945-0821-41b6-91f1-ba7b9956c183 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:23:38.594040 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 124/254] 10.222.0.33 () {60 vars in 1132 bytes} [Mon Jul 25 05:23:38 2022] POST /identity/v3/users => generated 293 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 05:23:38.594700 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9846c945-0821-41b6-91f1-ba7b9956c183 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:23:38.643959 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:23:38.673500 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c88082-a432-4568-bf6c-497960e4c9b0 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:23:38.712443 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa74818-b731-4fb5-b6f1-7d452e4c3c2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:23:38.713368 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa74818-b731-4fb5-b6f1-7d452e4c3c2c None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:23:38.713654 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa74818-b731-4fb5-b6f1-7d452e4c3c2c None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:23:38.713939 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa74818-b731-4fb5-b6f1-7d452e4c3c2c None admin] PATH_INFO: `/v3/projects/ec1ed227a23f453cab64e15036af0c60` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:23:38.715190 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfa74818-b731-4fb5-b6f1-7d452e4c3c2c None admin] RBAC: Authorizing `identity:get_project(project_id=ec1ed227a23f453cab64e15036af0c60)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:23:38.716657 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfa74818-b731-4fb5-b6f1-7d452e4c3c2c None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:23:38.719293 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 131/255] 10.222.0.33 () {56 vars in 1186 bytes} [Mon Jul 25 05:23:38 2022] GET /identity/v3/projects/ec1ed227a23f453cab64e15036af0c60 => generated 302 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:23:38.719293 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa74818-b731-4fb5-b6f1-7d452e4c3c2c None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:23:38.736010 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9846c945-0821-41b6-91f1-ba7b9956c183 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:23:38.752602 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:23:38.780351 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9903b83f-6122-4259-a064-902ddcc7a591 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:23:38.781396 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9903b83f-6122-4259-a064-902ddcc7a591 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:23:38.781686 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9903b83f-6122-4259-a064-902ddcc7a591 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:23:38.781967 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9903b83f-6122-4259-a064-902ddcc7a591 None admin] PATH_INFO: `/v3/projects/ec1ed227a23f453cab64e15036af0c60` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:23:38.784463 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9903b83f-6122-4259-a064-902ddcc7a591 None admin] RBAC: Authorizing `identity:get_project(project_id=ec1ed227a23f453cab64e15036af0c60)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:23:38.784647 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9903b83f-6122-4259-a064-902ddcc7a591 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:23:38.786419 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 125/256] 10.222.0.33 () {56 vars in 1186 bytes} [Mon Jul 25 05:23:38 2022] GET /identity/v3/projects/ec1ed227a23f453cab64e15036af0c60 => generated 302 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:23:38.790718 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3716fff1-e9b2-4d15-af87-7f8955321940 None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:23:38.790718 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3716fff1-e9b2-4d15-af87-7f8955321940 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:23:38.790718 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3716fff1-e9b2-4d15-af87-7f8955321940 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:23:38.808703 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-3716fff1-e9b2-4d15-af87-7f8955321940 None None] MFA Rules not processed for user `1eae540242d94c10b0ad33960a855f08`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:23:38.809964 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-3716fff1-e9b2-4d15-af87-7f8955321940 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:23:38.857712 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa74818-b731-4fb5-b6f1-7d452e4c3c2c None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:23:38.889176 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 126/257] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jul 25 05:23:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 05:23:38.890155 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3716fff1-e9b2-4d15-af87-7f8955321940 None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:23:38.903523 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd496a8c-1c37-4664-86b0-51d4e8c9a315 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:23:38.904363 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd496a8c-1c37-4664-86b0-51d4e8c9a315 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:23:38.904650 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd496a8c-1c37-4664-86b0-51d4e8c9a315 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:23:38.905757 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd496a8c-1c37-4664-86b0-51d4e8c9a315 None admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:23:38.906612 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd496a8c-1c37-4664-86b0-51d4e8c9a315 None admin] RBAC: Adding query filter params (name=member) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:23:38.906970 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd496a8c-1c37-4664-86b0-51d4e8c9a315 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:23:38.908517 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd496a8c-1c37-4664-86b0-51d4e8c9a315 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:23:38.919569 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 132/258] 10.222.0.33 () {56 vars in 1101 bytes} [Mon Jul 25 05:23:38 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:23:38.921216 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd496a8c-1c37-4664-86b0-51d4e8c9a315 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:23:38.931795 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:23:39.012064 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3716fff1-e9b2-4d15-af87-7f8955321940 None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:23:39.030350 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:23:39.040470 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd496a8c-1c37-4664-86b0-51d4e8c9a315 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:23:39.058399 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a1a3418-5269-4de7-9bcc-6913101c9130 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:23:39.059084 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1a3418-5269-4de7-9bcc-6913101c9130 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:23:39.059375 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1a3418-5269-4de7-9bcc-6913101c9130 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:23:39.059888 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1a3418-5269-4de7-9bcc-6913101c9130 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:23:39.061677 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a1a3418-5269-4de7-9bcc-6913101c9130 None admin] RBAC: Adding query filter params (role.id=39e5c666285a42fe8fc2c3afca6260d1, scope.project.id=ec1ed227a23f453cab64e15036af0c60, user.id=1c6dda23da0644c2b35c2b1ffc72a8f1) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:23:39.061993 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a1a3418-5269-4de7-9bcc-6913101c9130 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:23:39.063454 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a1a3418-5269-4de7-9bcc-6913101c9130 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:23:39.072992 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 127/259] 10.222.0.33 () {56 vars in 1374 bytes} [Mon Jul 25 05:23:38 2022] GET /identity/v3/role_assignments?user.id=1c6dda23da0644c2b35c2b1ffc72a8f1&scope.project.id=ec1ed227a23f453cab64e15036af0c60&role.id=39e5c666285a42fe8fc2c3afca6260d1 => generated 608 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:23:39.074007 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a1a3418-5269-4de7-9bcc-6913101c9130 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:23:39.080162 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ae58d5-9fa3-47f9-b9c2-deab0a59b313 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:23:39.080943 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ae58d5-9fa3-47f9-b9c2-deab0a59b313 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:23:39.081475 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ae58d5-9fa3-47f9-b9c2-deab0a59b313 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:23:39.081549 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ae58d5-9fa3-47f9-b9c2-deab0a59b313 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:23:39.082433 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33ae58d5-9fa3-47f9-b9c2-deab0a59b313 None admin] RBAC: Adding query filter params (role.id=be79956018b14baba7a021e2adf27084, scope.project.id=ec1ed227a23f453cab64e15036af0c60, user.id=e03df72adbd34b80951f6d287621b9b9) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:23:39.082758 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33ae58d5-9fa3-47f9-b9c2-deab0a59b313 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:23:39.084270 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33ae58d5-9fa3-47f9-b9c2-deab0a59b313 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:23:39.091446 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 133/260] 10.222.0.33 () {56 vars in 1374 bytes} [Mon Jul 25 05:23:38 2022] GET /identity/v3/role_assignments?user.id=e03df72adbd34b80951f6d287621b9b9&scope.project.id=ec1ed227a23f453cab64e15036af0c60&role.id=be79956018b14baba7a021e2adf27084 => generated 259 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:23:39.092869 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ae58d5-9fa3-47f9-b9c2-deab0a59b313 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:23:39.240179 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:23:39.288971 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a1a3418-5269-4de7-9bcc-6913101c9130 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:23:39.318137 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ae58d5-9fa3-47f9-b9c2-deab0a59b313 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:23:39.334274 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b89ef50b-677c-48ba-a8b8-a6cb978dc004 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:23:39.335342 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89ef50b-677c-48ba-a8b8-a6cb978dc004 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:23:39.335627 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89ef50b-677c-48ba-a8b8-a6cb978dc004 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:23:39.335928 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89ef50b-677c-48ba-a8b8-a6cb978dc004 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:23:39.351386 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b89ef50b-677c-48ba-a8b8-a6cb978dc004 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 05:23:39.362413 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application [None req-b89ef50b-677c-48ba-a8b8-a6cb978dc004 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 25 05:23:39.362413 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:23:39.362413 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:23:39.362413 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:23:39.362413 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:23:39.362413 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:23:39.362413 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:23:39.362413 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:23:39.362413 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:23:39.362413 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:23:39.362413 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:23:39.362413 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:23:39.362413 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 05:23:39.362413 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 05:23:39.362413 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 05:23:39.362413 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 05:23:39.363328 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 05:23:39.363328 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 05:23:39.363328 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 05:23:39.363328 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 05:23:39.363328 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:23:39.363328 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:23:39.363328 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:23:39.363328 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:23:39.363328 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:23:39.363328 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:23:39.363328 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:23:39.363328 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application with Lock( Jul 25 05:23:39.363328 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:23:39.363328 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:23:39.363328 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:23:39.363328 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:23:39.364120 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:23:39.364120 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:23:39.364120 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:23:39.364120 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:23:39.364120 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 05:23:39.364120 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 05:23:39.364120 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 05:23:39.364120 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 05:23:39.364120 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 05:23:39.364120 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 05:23:39.364120 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 25 05:23:39.364120 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Jul 25 05:23:39.364717 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f5e204-3dae-4fed-8475-5d0158fd148b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:23:39.364717 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f5e204-3dae-4fed-8475-5d0158fd148b None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:23:39.365457 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 128/261] 10.222.0.33 () {56 vars in 1114 bytes} [Mon Jul 25 05:23:39 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 291 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:23:39.365530 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f5e204-3dae-4fed-8475-5d0158fd148b None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:23:39.365530 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f5e204-3dae-4fed-8475-5d0158fd148b None admin] PATH_INFO: `/v3/users/0c23382cdaf5487993b90e76e5c60721` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:23:39.371014 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74f5e204-3dae-4fed-8475-5d0158fd148b None admin] RBAC: Authorizing `identity:get_user(user_id=0c23382cdaf5487993b90e76e5c60721)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:23:39.371185 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b89ef50b-677c-48ba-a8b8-a6cb978dc004 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:23:39.372542 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74f5e204-3dae-4fed-8475-5d0158fd148b None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:23:39.374761 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 134/262] 10.222.0.33 () {56 vars in 1177 bytes} [Mon Jul 25 05:23:39 2022] GET /identity/v3/users/0c23382cdaf5487993b90e76e5c60721 => generated 297 bytes in 283 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:23:39.380101 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f5e204-3dae-4fed-8475-5d0158fd148b None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:23:39.441796 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:23:39.560169 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b89ef50b-677c-48ba-a8b8-a6cb978dc004 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:23:39.573379 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f5e204-3dae-4fed-8475-5d0158fd148b None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:23:39.607827 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e7836de-9fe3-46c7-8776-2bf963951457 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:23:39.608556 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7836de-9fe3-46c7-8776-2bf963951457 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:23:39.608873 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7836de-9fe3-46c7-8776-2bf963951457 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:23:39.609155 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7836de-9fe3-46c7-8776-2bf963951457 None admin] PATH_INFO: `/v3/roles` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:23:39.609931 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e7836de-9fe3-46c7-8776-2bf963951457 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:23:39.610241 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e7836de-9fe3-46c7-8776-2bf963951457 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:23:39.611654 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e7836de-9fe3-46c7-8776-2bf963951457 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:23:39.630779 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 129/263] 10.222.0.33 () {56 vars in 1111 bytes} [Mon Jul 25 05:23:39 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:23:39.637644 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e7836de-9fe3-46c7-8776-2bf963951457 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:23:39.654383 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f2bb12-26a8-4d1a-9414-e068c434d417 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:23:39.655858 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f2bb12-26a8-4d1a-9414-e068c434d417 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:23:39.656147 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f2bb12-26a8-4d1a-9414-e068c434d417 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:23:39.656428 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f2bb12-26a8-4d1a-9414-e068c434d417 None admin] PATH_INFO: `/v3/projects/ec1ed227a23f453cab64e15036af0c60` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:23:39.659179 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0f2bb12-26a8-4d1a-9414-e068c434d417 None admin] RBAC: Authorizing `identity:get_project(project_id=ec1ed227a23f453cab64e15036af0c60)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:23:39.660663 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0f2bb12-26a8-4d1a-9414-e068c434d417 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:23:39.662511 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 135/264] 10.222.0.33 () {56 vars in 1186 bytes} [Mon Jul 25 05:23:39 2022] GET /identity/v3/projects/ec1ed227a23f453cab64e15036af0c60 => generated 302 bytes in 283 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 05:23:39.676319 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f2bb12-26a8-4d1a-9414-e068c434d417 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:23:39.691032 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:23:39.798426 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e7836de-9fe3-46c7-8776-2bf963951457 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:23:39.806374 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:23:39.834639 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e25b605-27c5-4c56-9c40-e056bb464403 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:23:39.835644 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e25b605-27c5-4c56-9c40-e056bb464403 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:23:39.835932 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e25b605-27c5-4c56-9c40-e056bb464403 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:23:39.836213 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e25b605-27c5-4c56-9c40-e056bb464403 None admin] PATH_INFO: `/v3/users/1c6dda23da0644c2b35c2b1ffc72a8f1` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:23:39.837443 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e25b605-27c5-4c56-9c40-e056bb464403 None admin] RBAC: Authorizing `identity:get_user(user_id=1c6dda23da0644c2b35c2b1ffc72a8f1)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:23:39.838955 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e25b605-27c5-4c56-9c40-e056bb464403 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:23:39.841136 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 130/265] 10.222.0.33 () {56 vars in 1177 bytes} [Mon Jul 25 05:23:39 2022] GET /identity/v3/users/1c6dda23da0644c2b35c2b1ffc72a8f1 => generated 283 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:23:39.848642 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e25b605-27c5-4c56-9c40-e056bb464403 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:23:39.909037 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f2bb12-26a8-4d1a-9414-e068c434d417 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:23:39.978454 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-479a7c2b-18aa-47ec-a6b7-3098ae76fd1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:23:39.979586 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479a7c2b-18aa-47ec-a6b7-3098ae76fd1b None admin] REQUEST_METHOD: `PUT` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:23:39.979859 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479a7c2b-18aa-47ec-a6b7-3098ae76fd1b None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:23:39.980137 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479a7c2b-18aa-47ec-a6b7-3098ae76fd1b None admin] PATH_INFO: `/v3/projects/ec1ed227a23f453cab64e15036af0c60/users/0c23382cdaf5487993b90e76e5c60721/roles/5074e74e35f54d4b9bb415c34999a82a` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:23:39.983955 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-479a7c2b-18aa-47ec-a6b7-3098ae76fd1b None admin] RBAC: Authorizing `identity:create_grant(project_id=ec1ed227a23f453cab64e15036af0c60, user_id=0c23382cdaf5487993b90e76e5c60721, role_id=5074e74e35f54d4b9bb415c34999a82a)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:23:39.985461 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-479a7c2b-18aa-47ec-a6b7-3098ae76fd1b None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:23:40.012376 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 136/266] 10.222.0.33 () {58 vars in 1439 bytes} [Mon Jul 25 05:23:39 2022] PUT /identity/v3/projects/ec1ed227a23f453cab64e15036af0c60/users/0c23382cdaf5487993b90e76e5c60721/roles/5074e74e35f54d4b9bb415c34999a82a => generated 0 bytes in 337 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 25 05:23:40.041099 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e25b605-27c5-4c56-9c40-e056bb464403 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:23:40.082308 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aadc02c-2544-4616-ba9a-6852b79a573e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:23:40.083021 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aadc02c-2544-4616-ba9a-6852b79a573e None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:23:40.083416 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aadc02c-2544-4616-ba9a-6852b79a573e None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:23:40.083762 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aadc02c-2544-4616-ba9a-6852b79a573e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:23:40.122523 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:23:40.176692 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aadc02c-2544-4616-ba9a-6852b79a573e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:23:40.178208 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aadc02c-2544-4616-ba9a-6852b79a573e None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:23:40.289845 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 131/267] 10.222.0.33 () {58 vars in 1282 bytes} [Mon Jul 25 05:23:39 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 442 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 25 05:23:40.296294 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-479a7c2b-18aa-47ec-a6b7-3098ae76fd1b None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:23:40.400211 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:23:40.441321 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-479a7c2b-18aa-47ec-a6b7-3098ae76fd1b None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:23:40.490863 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3051c3-b831-44e9-8d92-5f58f7772ae7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:23:40.491732 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3051c3-b831-44e9-8d92-5f58f7772ae7 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:23:40.492019 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3051c3-b831-44e9-8d92-5f58f7772ae7 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:23:40.492300 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3051c3-b831-44e9-8d92-5f58f7772ae7 None admin] PATH_INFO: `/v3/projects/ec1ed227a23f453cab64e15036af0c60` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:23:40.493576 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc3051c3-b831-44e9-8d92-5f58f7772ae7 None admin] RBAC: Authorizing `identity:get_project(project_id=ec1ed227a23f453cab64e15036af0c60)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:23:40.495116 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc3051c3-b831-44e9-8d92-5f58f7772ae7 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:23:40.497194 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 137/268] 10.222.0.33 () {56 vars in 1186 bytes} [Mon Jul 25 05:23:40 2022] GET /identity/v3/projects/ec1ed227a23f453cab64e15036af0c60 => generated 302 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:23:40.503470 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aadc02c-2544-4616-ba9a-6852b79a573e None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:23:40.580803 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:23:40.626172 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aadc02c-2544-4616-ba9a-6852b79a573e None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:23:40.674593 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d2ebe1a-c7fd-47fa-84ad-242a9e5a9916 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:23:40.675235 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2ebe1a-c7fd-47fa-84ad-242a9e5a9916 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:23:40.675522 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2ebe1a-c7fd-47fa-84ad-242a9e5a9916 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:23:40.675798 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2ebe1a-c7fd-47fa-84ad-242a9e5a9916 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:23:40.676631 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d2ebe1a-c7fd-47fa-84ad-242a9e5a9916 None admin] RBAC: Adding query filter params (role.id=02aba1f7cbdf4d6595e18af7fb5727e0, scope.project.id=ec1ed227a23f453cab64e15036af0c60, user.id=1c6dda23da0644c2b35c2b1ffc72a8f1) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:23:40.676966 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d2ebe1a-c7fd-47fa-84ad-242a9e5a9916 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:23:40.678409 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d2ebe1a-c7fd-47fa-84ad-242a9e5a9916 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:23:40.687665 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 132/269] 10.222.0.33 () {56 vars in 1374 bytes} [Mon Jul 25 05:23:40 2022] GET /identity/v3/role_assignments?user.id=1c6dda23da0644c2b35c2b1ffc72a8f1&scope.project.id=ec1ed227a23f453cab64e15036af0c60&role.id=02aba1f7cbdf4d6595e18af7fb5727e0 => generated 608 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:23:41.284233 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b9a0d2-6bfa-46b1-a788-da195bcf8aed None None] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:23:41.284523 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b9a0d2-6bfa-46b1-a788-da195bcf8aed None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:23:41.285083 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b9a0d2-6bfa-46b1-a788-da195bcf8aed None None] PATH_INFO: `/` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:23:41.286963 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 138/270] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jul 25 05:23:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:23:41.292245 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567b30a0-7ddb-4658-9163-1b63658507cf None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:23:41.293125 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567b30a0-7ddb-4658-9163-1b63658507cf None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:23:41.293608 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567b30a0-7ddb-4658-9163-1b63658507cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:23:41.311959 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-567b30a0-7ddb-4658-9163-1b63658507cf None None] MFA Rules not processed for user `1eae540242d94c10b0ad33960a855f08`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:23:41.313541 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-567b30a0-7ddb-4658-9163-1b63658507cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:23:41.317615 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74b3fbe-7c67-4df9-97d8-c4decf0b85c6 None None] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:23:41.318500 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74b3fbe-7c67-4df9-97d8-c4decf0b85c6 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:23:41.318985 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74b3fbe-7c67-4df9-97d8-c4decf0b85c6 None None] PATH_INFO: `/` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:23:41.320583 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 139/271] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jul 25 05:23:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:23:41.325806 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc0f656-33f4-4f22-a7b0-7be31150c035 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:23:41.326330 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc0f656-33f4-4f22-a7b0-7be31150c035 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:23:41.326844 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc0f656-33f4-4f22-a7b0-7be31150c035 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:23:41.344400 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-ccc0f656-33f4-4f22-a7b0-7be31150c035 None None] MFA Rules not processed for user `1eae540242d94c10b0ad33960a855f08`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:23:41.346018 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-ccc0f656-33f4-4f22-a7b0-7be31150c035 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:23:41.384122 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 133/272] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jul 25 05:23:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 05:23:41.408704 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-567b30a0-7ddb-4658-9163-1b63658507cf None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:23:41.417433 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 140/273] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jul 25 05:23:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 25 05:23:41.442881 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc0f656-33f4-4f22-a7b0-7be31150c035 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:23:41.473383 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:23:41.520429 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:23:41.528069 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-567b30a0-7ddb-4658-9163-1b63658507cf None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:23:41.562040 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc0f656-33f4-4f22-a7b0-7be31150c035 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:23:41.566702 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bedcce4-43f8-42f2-b8fe-2bdf0595a6d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:23:41.567503 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bedcce4-43f8-42f2-b8fe-2bdf0595a6d4 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:23:41.567785 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bedcce4-43f8-42f2-b8fe-2bdf0595a6d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:23:41.568063 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bedcce4-43f8-42f2-b8fe-2bdf0595a6d4 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:23:41.578809 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bedcce4-43f8-42f2-b8fe-2bdf0595a6d4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 05:23:41.586691 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application [None req-5bedcce4-43f8-42f2-b8fe-2bdf0595a6d4 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 25 05:23:41.586691 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:23:41.586691 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:23:41.586691 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:23:41.586691 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:23:41.586691 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:23:41.586691 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:23:41.586691 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:23:41.586691 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:23:41.586691 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:23:41.586691 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:23:41.586691 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:23:41.586691 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 05:23:41.586691 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 05:23:41.586691 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 05:23:41.586691 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 05:23:41.587562 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 05:23:41.587562 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 05:23:41.587562 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 05:23:41.587562 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 05:23:41.587562 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:23:41.587562 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:23:41.587562 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:23:41.587562 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:23:41.587562 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:23:41.587562 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:23:41.587562 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:23:41.587562 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application with Lock( Jul 25 05:23:41.587562 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:23:41.587562 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:23:41.587562 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:23:41.587562 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:23:41.588335 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:23:41.588335 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:23:41.588335 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:23:41.588335 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:23:41.588335 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 05:23:41.588335 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 05:23:41.588335 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 05:23:41.588335 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 05:23:41.588335 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 05:23:41.588335 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 05:23:41.588335 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 25 05:23:41.588335 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Jul 25 05:23:41.588335 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 134/274] 10.222.0.33 () {56 vars in 1099 bytes} [Mon Jul 25 05:23:41 2022] GET /identity/v3/roles/member => generated 84 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 25 05:23:41.589628 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9ac1d5-d683-4e60-82f3-e8b80011a6dd None None] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:23:41.589628 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9ac1d5-d683-4e60-82f3-e8b80011a6dd None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:23:41.589903 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9ac1d5-d683-4e60-82f3-e8b80011a6dd None None] PATH_INFO: `/` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:23:41.591428 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 135/275] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jul 25 05:23:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:23:41.596286 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a9ac1d5-d683-4e60-82f3-e8b80011a6dd None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:23:41.603409 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa8ae110-db8d-46bf-822f-f5aaaa950745 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:23:41.604210 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8ae110-db8d-46bf-822f-f5aaaa950745 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:23:41.604497 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8ae110-db8d-46bf-822f-f5aaaa950745 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:23:41.604795 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8ae110-db8d-46bf-822f-f5aaaa950745 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:23:41.605982 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa8ae110-db8d-46bf-822f-f5aaaa950745 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:23:41.607445 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa8ae110-db8d-46bf-822f-f5aaaa950745 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:23:41.610027 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 141/276] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Jul 25 05:23:41 2022] GET /identity/v3/domains/default => generated 199 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 05:23:41.611168 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021ae62c-2212-477c-9d35-0e28a72f0c8f None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:23:41.611457 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021ae62c-2212-477c-9d35-0e28a72f0c8f None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:23:41.611737 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021ae62c-2212-477c-9d35-0e28a72f0c8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:23:41.628238 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-021ae62c-2212-477c-9d35-0e28a72f0c8f None None] MFA Rules not processed for user `1eae540242d94c10b0ad33960a855f08`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:23:41.629455 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-021ae62c-2212-477c-9d35-0e28a72f0c8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:23:41.704141 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 142/277] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jul 25 05:23:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 05:23:41.705035 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-021ae62c-2212-477c-9d35-0e28a72f0c8f None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:23:41.724173 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a9ac1d5-d683-4e60-82f3-e8b80011a6dd None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:23:41.746312 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:23:41.762013 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f81b3e-d619-466d-ab78-0c70c2c3a303 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:23:41.762686 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f81b3e-d619-466d-ab78-0c70c2c3a303 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:23:41.762974 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f81b3e-d619-466d-ab78-0c70c2c3a303 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:23:41.763404 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f81b3e-d619-466d-ab78-0c70c2c3a303 None admin] PATH_INFO: `/v3/roles` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:23:41.764099 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6f81b3e-d619-466d-ab78-0c70c2c3a303 None admin] RBAC: Adding query filter params (name=member) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:23:41.764889 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6f81b3e-d619-466d-ab78-0c70c2c3a303 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:23:41.766273 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6f81b3e-d619-466d-ab78-0c70c2c3a303 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:23:41.769409 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:23:41.779441 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 136/278] 10.222.0.33 () {56 vars in 1101 bytes} [Mon Jul 25 05:23:41 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:23:41.779441 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f81b3e-d619-466d-ab78-0c70c2c3a303 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:23:41.824399 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-021ae62c-2212-477c-9d35-0e28a72f0c8f None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:23:41.865249 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e02f57f-12b7-4946-90e9-a85f217787a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:23:41.865932 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e02f57f-12b7-4946-90e9-a85f217787a5 None admin] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:23:41.866221 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e02f57f-12b7-4946-90e9-a85f217787a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:23:41.866502 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e02f57f-12b7-4946-90e9-a85f217787a5 None admin] PATH_INFO: `/v3/users` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:23:41.867218 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e02f57f-12b7-4946-90e9-a85f217787a5 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:23:41.868657 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e02f57f-12b7-4946-90e9-a85f217787a5 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:23:41.890265 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f81b3e-d619-466d-ab78-0c70c2c3a303 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:23:41.891533 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 143/279] 10.222.0.33 () {60 vars in 1132 bytes} [Mon Jul 25 05:23:41 2022] POST /identity/v3/users => generated 293 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 05:23:41.893574 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e02f57f-12b7-4946-90e9-a85f217787a5 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:23:41.937093 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431b45de-1fdb-43d5-8ba3-65136cc9ede9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:23:41.938357 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431b45de-1fdb-43d5-8ba3-65136cc9ede9 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:23:41.939006 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431b45de-1fdb-43d5-8ba3-65136cc9ede9 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:23:41.939723 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431b45de-1fdb-43d5-8ba3-65136cc9ede9 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:23:41.956823 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-431b45de-1fdb-43d5-8ba3-65136cc9ede9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 05:23:41.968251 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application [None req-431b45de-1fdb-43d5-8ba3-65136cc9ede9 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 25 05:23:41.968251 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:23:41.968251 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:23:41.968251 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:23:41.968251 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:23:41.968251 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:23:41.968251 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:23:41.968251 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:23:41.968251 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:23:41.968251 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:23:41.968251 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:23:41.968251 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:23:41.968251 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 05:23:41.968251 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 05:23:41.968251 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 05:23:41.968251 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 05:23:41.969289 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 05:23:41.969289 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 05:23:41.969289 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 05:23:41.969289 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 05:23:41.969289 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:23:41.969289 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:23:41.969289 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:23:41.969289 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:23:41.969289 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:23:41.969289 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:23:41.969289 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:23:41.969289 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application with Lock( Jul 25 05:23:41.969289 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:23:41.969289 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:23:41.969289 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:23:41.969289 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:23:41.970113 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:23:41.970113 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:23:41.970113 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:23:41.970113 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:23:41.970113 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 05:23:41.970113 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 05:23:41.970113 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 05:23:41.970113 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 05:23:41.970113 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 05:23:41.970113 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 05:23:41.970113 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 25 05:23:41.970113 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Jul 25 05:23:41.972767 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 137/280] 10.222.0.33 () {56 vars in 1099 bytes} [Mon Jul 25 05:23:41 2022] GET /identity/v3/roles/reader => generated 84 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:23:41.984706 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431b45de-1fdb-43d5-8ba3-65136cc9ede9 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:23:42.040910 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:23:42.050020 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e02f57f-12b7-4946-90e9-a85f217787a5 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:23:42.073041 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:23:42.096205 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67de71e7-d840-4d7a-a0e7-bef8012846e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:23:42.097490 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67de71e7-d840-4d7a-a0e7-bef8012846e1 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:23:42.098058 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67de71e7-d840-4d7a-a0e7-bef8012846e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:23:42.098346 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67de71e7-d840-4d7a-a0e7-bef8012846e1 None admin] PATH_INFO: `/v3/users/656c60432bfb4d3e95b1250387de6b06` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:23:42.110581 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431b45de-1fdb-43d5-8ba3-65136cc9ede9 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:23:42.116687 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67de71e7-d840-4d7a-a0e7-bef8012846e1 None admin] RBAC: Authorizing `identity:get_user(user_id=656c60432bfb4d3e95b1250387de6b06)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:23:42.118194 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67de71e7-d840-4d7a-a0e7-bef8012846e1 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:23:42.120568 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 144/281] 10.222.0.33 () {56 vars in 1177 bytes} [Mon Jul 25 05:23:41 2022] GET /identity/v3/users/656c60432bfb4d3e95b1250387de6b06 => generated 293 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:23:42.127621 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67de71e7-d840-4d7a-a0e7-bef8012846e1 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:23:42.158643 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d75f6f-3a61-47db-afb0-59ba1cd03cc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:23:42.159484 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d75f6f-3a61-47db-afb0-59ba1cd03cc4 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:23:42.159773 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d75f6f-3a61-47db-afb0-59ba1cd03cc4 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:23:42.160385 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d75f6f-3a61-47db-afb0-59ba1cd03cc4 None admin] PATH_INFO: `/v3/roles` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:23:42.161514 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87d75f6f-3a61-47db-afb0-59ba1cd03cc4 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:23:42.163847 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87d75f6f-3a61-47db-afb0-59ba1cd03cc4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:23:42.163960 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87d75f6f-3a61-47db-afb0-59ba1cd03cc4 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:23:42.175771 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 138/282] 10.222.0.33 () {56 vars in 1101 bytes} [Mon Jul 25 05:23:41 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:23:42.181140 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d75f6f-3a61-47db-afb0-59ba1cd03cc4 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:23:42.255811 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67de71e7-d840-4d7a-a0e7-bef8012846e1 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:23:42.302789 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6357dbbd-73ad-4b43-844f-bc0b99bc27a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:23:42.303731 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6357dbbd-73ad-4b43-844f-bc0b99bc27a0 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:23:42.304021 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6357dbbd-73ad-4b43-844f-bc0b99bc27a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:23:42.304306 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6357dbbd-73ad-4b43-844f-bc0b99bc27a0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:23:42.305186 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6357dbbd-73ad-4b43-844f-bc0b99bc27a0 None admin] RBAC: Adding query filter params (role.id=5074e74e35f54d4b9bb415c34999a82a, user.id=656c60432bfb4d3e95b1250387de6b06, scope.system=all) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:23:42.305668 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6357dbbd-73ad-4b43-844f-bc0b99bc27a0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:23:42.307455 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6357dbbd-73ad-4b43-844f-bc0b99bc27a0 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:23:42.313999 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 145/283] 10.222.0.33 () {56 vars in 1308 bytes} [Mon Jul 25 05:23:42 2022] GET /identity/v3/role_assignments?user.id=656c60432bfb4d3e95b1250387de6b06&scope.system=all&role.id=5074e74e35f54d4b9bb415c34999a82a => generated 226 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 05:23:42.323279 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d75f6f-3a61-47db-afb0-59ba1cd03cc4 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:23:42.328502 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:23:42.375293 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6436fa2-31a1-4819-a983-73ec6e6903c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:23:42.376136 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6436fa2-31a1-4819-a983-73ec6e6903c2 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:23:42.376422 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6436fa2-31a1-4819-a983-73ec6e6903c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:23:42.376699 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6436fa2-31a1-4819-a983-73ec6e6903c2 None admin] PATH_INFO: `/v3/users/e03df72adbd34b80951f6d287621b9b9` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:23:42.379031 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6436fa2-31a1-4819-a983-73ec6e6903c2 None admin] RBAC: Authorizing `identity:get_user(user_id=e03df72adbd34b80951f6d287621b9b9)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:23:42.380553 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6436fa2-31a1-4819-a983-73ec6e6903c2 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:23:42.382735 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 139/284] 10.222.0.33 () {56 vars in 1177 bytes} [Mon Jul 25 05:23:42 2022] GET /identity/v3/users/e03df72adbd34b80951f6d287621b9b9 => generated 297 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:23:42.388477 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6357dbbd-73ad-4b43-844f-bc0b99bc27a0 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:23:42.443631 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:23:42.463109 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f756e2f4-24d1-47d0-aaa4-7f91291a1287 None None] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:23:42.463457 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f756e2f4-24d1-47d0-aaa4-7f91291a1287 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:23:42.463672 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f756e2f4-24d1-47d0-aaa4-7f91291a1287 None None] PATH_INFO: `/` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:23:42.465792 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 140/285] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jul 25 05:23:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:23:42.472278 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28c6a58-3a5d-49d3-ac29-e50054c708d6 None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:23:42.472566 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28c6a58-3a5d-49d3-ac29-e50054c708d6 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:23:42.472877 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28c6a58-3a5d-49d3-ac29-e50054c708d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:23:42.491828 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-f28c6a58-3a5d-49d3-ac29-e50054c708d6 None None] MFA Rules not processed for user `1eae540242d94c10b0ad33960a855f08`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:23:42.493078 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-f28c6a58-3a5d-49d3-ac29-e50054c708d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:23:42.549589 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6357dbbd-73ad-4b43-844f-bc0b99bc27a0 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:23:42.566900 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 141/286] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jul 25 05:23:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 25 05:23:42.590002 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f28c6a58-3a5d-49d3-ac29-e50054c708d6 None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:23:42.600395 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f444dfcc-a36c-486c-8b19-78d2ac366d15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:23:42.601301 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f444dfcc-a36c-486c-8b19-78d2ac366d15 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:23:42.601589 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f444dfcc-a36c-486c-8b19-78d2ac366d15 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:23:42.601868 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f444dfcc-a36c-486c-8b19-78d2ac366d15 None admin] PATH_INFO: `/v3/projects/ec1ed227a23f453cab64e15036af0c60` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:23:42.605508 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f444dfcc-a36c-486c-8b19-78d2ac366d15 None admin] RBAC: Authorizing `identity:get_project(project_id=ec1ed227a23f453cab64e15036af0c60)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:23:42.606968 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f444dfcc-a36c-486c-8b19-78d2ac366d15 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:23:42.610097 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 146/287] 10.222.0.33 () {56 vars in 1186 bytes} [Mon Jul 25 05:23:42 2022] GET /identity/v3/projects/ec1ed227a23f453cab64e15036af0c60 => generated 302 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:23:42.618297 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f444dfcc-a36c-486c-8b19-78d2ac366d15 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:23:42.699343 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f28c6a58-3a5d-49d3-ac29-e50054c708d6 None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:23:42.700753 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:23:42.745970 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f444dfcc-a36c-486c-8b19-78d2ac366d15 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:23:42.746409 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a77ece-9428-4d74-acdf-05cb27102e4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:23:42.747447 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a77ece-9428-4d74-acdf-05cb27102e4f None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:23:42.747921 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a77ece-9428-4d74-acdf-05cb27102e4f None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:23:42.748472 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a77ece-9428-4d74-acdf-05cb27102e4f None admin] PATH_INFO: `/v3/roles/member` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:23:42.756498 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:23:42.773763 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38a77ece-9428-4d74-acdf-05cb27102e4f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 05:23:42.785046 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85d856b-20e4-4d58-a15a-09106470dafe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:23:42.786209 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85d856b-20e4-4d58-a15a-09106470dafe None admin] REQUEST_METHOD: `PUT` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:23:42.786698 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85d856b-20e4-4d58-a15a-09106470dafe None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:23:42.787218 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85d856b-20e4-4d58-a15a-09106470dafe None admin] PATH_INFO: `/v3/projects/ec1ed227a23f453cab64e15036af0c60/users/e03df72adbd34b80951f6d287621b9b9/roles/be79956018b14baba7a021e2adf27084` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:23:42.789476 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a85d856b-20e4-4d58-a15a-09106470dafe None admin] RBAC: Authorizing `identity:create_grant(project_id=ec1ed227a23f453cab64e15036af0c60, user_id=e03df72adbd34b80951f6d287621b9b9, role_id=be79956018b14baba7a021e2adf27084)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:23:42.790232 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application [None req-38a77ece-9428-4d74-acdf-05cb27102e4f None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 25 05:23:42.790232 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:23:42.790232 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:23:42.790232 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:23:42.790232 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:23:42.790232 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:23:42.790232 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:23:42.790232 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:23:42.790232 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:23:42.790232 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:23:42.790232 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:23:42.790232 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:23:42.790232 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 05:23:42.790232 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 05:23:42.790232 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 05:23:42.790232 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 05:23:42.791103 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 05:23:42.791103 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 05:23:42.791103 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 05:23:42.791103 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 05:23:42.791103 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:23:42.791103 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:23:42.791103 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:23:42.791103 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:23:42.791103 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:23:42.791103 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:23:42.791103 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:23:42.791103 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application with Lock( Jul 25 05:23:42.791103 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:23:42.791103 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:23:42.791103 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:23:42.791103 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:23:42.791905 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:23:42.791905 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:23:42.791905 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:23:42.791905 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:23:42.791905 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 05:23:42.791905 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 05:23:42.791905 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 05:23:42.791905 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 05:23:42.791905 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 05:23:42.791905 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 05:23:42.791905 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 25 05:23:42.791905 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Jul 25 05:23:42.791905 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 142/288] 10.222.0.33 () {56 vars in 1099 bytes} [Mon Jul 25 05:23:42 2022] GET /identity/v3/roles/member => generated 84 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:23:42.794058 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a85d856b-20e4-4d58-a15a-09106470dafe None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:23:42.808412 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a77ece-9428-4d74-acdf-05cb27102e4f None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:23:42.812526 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 147/289] 10.222.0.33 () {58 vars in 1439 bytes} [Mon Jul 25 05:23:42 2022] PUT /identity/v3/projects/ec1ed227a23f453cab64e15036af0c60/users/e03df72adbd34b80951f6d287621b9b9/roles/be79956018b14baba7a021e2adf27084 => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 05:23:43.049536 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a77ece-9428-4d74-acdf-05cb27102e4f None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:23:43.089942 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c4efad-a3c7-40cb-8041-86afa163d249 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:23:43.091010 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c4efad-a3c7-40cb-8041-86afa163d249 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:23:43.091521 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c4efad-a3c7-40cb-8041-86afa163d249 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:23:43.092123 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c4efad-a3c7-40cb-8041-86afa163d249 None admin] PATH_INFO: `/v3/roles` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:23:43.093682 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23c4efad-a3c7-40cb-8041-86afa163d249 None admin] RBAC: Adding query filter params (name=member) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:23:43.094207 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23c4efad-a3c7-40cb-8041-86afa163d249 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:23:43.096154 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23c4efad-a3c7-40cb-8041-86afa163d249 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:23:43.105304 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 143/290] 10.222.0.33 () {56 vars in 1101 bytes} [Mon Jul 25 05:23:42 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 297 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:23:43.113906 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85d856b-20e4-4d58-a15a-09106470dafe None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:23:43.343296 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85d856b-20e4-4d58-a15a-09106470dafe None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:23:43.401490 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae020a2f-fce2-46b8-86f7-78177981c7e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:23:43.403167 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae020a2f-fce2-46b8-86f7-78177981c7e3 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:23:43.403649 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae020a2f-fce2-46b8-86f7-78177981c7e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:23:43.404118 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae020a2f-fce2-46b8-86f7-78177981c7e3 None admin] PATH_INFO: `/v3/users/0c23382cdaf5487993b90e76e5c60721` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:23:43.409534 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae020a2f-fce2-46b8-86f7-78177981c7e3 None admin] RBAC: Authorizing `identity:get_user(user_id=0c23382cdaf5487993b90e76e5c60721)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:23:43.411232 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae020a2f-fce2-46b8-86f7-78177981c7e3 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:23:43.413158 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 148/291] 10.222.0.33 () {56 vars in 1177 bytes} [Mon Jul 25 05:23:43 2022] GET /identity/v3/users/0c23382cdaf5487993b90e76e5c60721 => generated 297 bytes in 300 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:23:43.418709 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c4efad-a3c7-40cb-8041-86afa163d249 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:23:43.465678 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:23:43.623687 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c4efad-a3c7-40cb-8041-86afa163d249 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:23:43.694806 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b3e6261-5d6d-4b43-ba08-b8480782212c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:23:43.695467 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3e6261-5d6d-4b43-ba08-b8480782212c None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:23:43.695758 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3e6261-5d6d-4b43-ba08-b8480782212c None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:23:43.696041 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3e6261-5d6d-4b43-ba08-b8480782212c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:23:43.788951 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b3e6261-5d6d-4b43-ba08-b8480782212c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:23:43.790757 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b3e6261-5d6d-4b43-ba08-b8480782212c None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:23:43.839982 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:23:43.945775 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 144/292] 10.222.0.33 () {58 vars in 1282 bytes} [Mon Jul 25 05:23:43 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 527 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 25 05:23:43.951948 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae020a2f-fce2-46b8-86f7-78177981c7e3 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:23:44.005485 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:23:44.082500 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae020a2f-fce2-46b8-86f7-78177981c7e3 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:23:44.120251 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-479e11c3-a224-4a64-babe-c60c1b4ed584 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:23:44.121135 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479e11c3-a224-4a64-babe-c60c1b4ed584 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:23:44.121418 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479e11c3-a224-4a64-babe-c60c1b4ed584 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:23:44.121692 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479e11c3-a224-4a64-babe-c60c1b4ed584 None admin] PATH_INFO: `/v3/projects/ec1ed227a23f453cab64e15036af0c60` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:23:44.124863 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-479e11c3-a224-4a64-babe-c60c1b4ed584 None admin] RBAC: Authorizing `identity:get_project(project_id=ec1ed227a23f453cab64e15036af0c60)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:23:44.126319 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-479e11c3-a224-4a64-babe-c60c1b4ed584 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:23:44.128044 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 149/293] 10.222.0.33 () {56 vars in 1186 bytes} [Mon Jul 25 05:23:43 2022] GET /identity/v3/projects/ec1ed227a23f453cab64e15036af0c60 => generated 302 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:23:44.133911 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b3e6261-5d6d-4b43-ba08-b8480782212c None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:23:44.258249 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b3e6261-5d6d-4b43-ba08-b8480782212c None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:23:44.296274 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:23:44.307850 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c1e9ea-9564-4e0d-b0cd-453887939959 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:23:44.313535 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c1e9ea-9564-4e0d-b0cd-453887939959 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:23:44.313825 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c1e9ea-9564-4e0d-b0cd-453887939959 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:23:44.314104 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c1e9ea-9564-4e0d-b0cd-453887939959 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:23:44.315204 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5c1e9ea-9564-4e0d-b0cd-453887939959 None admin] RBAC: Adding query filter params (role.id=5074e74e35f54d4b9bb415c34999a82a, scope.project.id=ec1ed227a23f453cab64e15036af0c60, user.id=0c23382cdaf5487993b90e76e5c60721) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:23:44.315516 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5c1e9ea-9564-4e0d-b0cd-453887939959 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:23:44.316982 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5c1e9ea-9564-4e0d-b0cd-453887939959 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:23:44.323263 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 145/294] 10.222.0.33 () {56 vars in 1374 bytes} [Mon Jul 25 05:23:44 2022] GET /identity/v3/role_assignments?user.id=0c23382cdaf5487993b90e76e5c60721&scope.project.id=ec1ed227a23f453cab64e15036af0c60&role.id=5074e74e35f54d4b9bb415c34999a82a => generated 608 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:23:44.567110 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e95f82-1a3d-43dd-82dd-671c4d40b693 None None] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:23:44.567861 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e95f82-1a3d-43dd-82dd-671c4d40b693 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:23:44.567861 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e95f82-1a3d-43dd-82dd-671c4d40b693 None None] PATH_INFO: `/` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:23:44.569339 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 150/295] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jul 25 05:23:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:23:44.575139 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83f39da-8e61-46ed-8a34-2c1d847758ef None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:23:44.575585 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83f39da-8e61-46ed-8a34-2c1d847758ef None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:23:44.575684 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83f39da-8e61-46ed-8a34-2c1d847758ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:23:44.597123 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-a83f39da-8e61-46ed-8a34-2c1d847758ef None None] MFA Rules not processed for user `1eae540242d94c10b0ad33960a855f08`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:23:44.599288 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-a83f39da-8e61-46ed-8a34-2c1d847758ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:23:44.679668 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 146/296] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jul 25 05:23:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 05:23:44.703687 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e95f82-1a3d-43dd-82dd-671c4d40b693 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:23:44.822509 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e95f82-1a3d-43dd-82dd-671c4d40b693 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:23:44.841149 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:23:44.865827 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd1cd7b-6e59-4ca6-91b1-823b63225932 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:23:44.866632 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd1cd7b-6e59-4ca6-91b1-823b63225932 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:23:44.866914 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd1cd7b-6e59-4ca6-91b1-823b63225932 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:23:44.867190 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd1cd7b-6e59-4ca6-91b1-823b63225932 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:23:44.879159 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cd1cd7b-6e59-4ca6-91b1-823b63225932 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 05:23:44.887302 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application [None req-8cd1cd7b-6e59-4ca6-91b1-823b63225932 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 25 05:23:44.887302 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:23:44.887302 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:23:44.887302 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:23:44.887302 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:23:44.887302 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:23:44.887302 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:23:44.887302 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:23:44.887302 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:23:44.887302 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:23:44.887302 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:23:44.887302 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:23:44.887302 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 05:23:44.887302 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 05:23:44.887302 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 05:23:44.887302 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 05:23:44.888175 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 05:23:44.888175 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 05:23:44.888175 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 05:23:44.888175 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 05:23:44.888175 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:23:44.888175 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:23:44.888175 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:23:44.888175 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:23:44.888175 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:23:44.888175 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:23:44.888175 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:23:44.888175 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application with Lock( Jul 25 05:23:44.888175 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:23:44.888175 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:23:44.888175 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:23:44.888175 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:23:44.889047 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:23:44.889047 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:23:44.889047 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:23:44.889047 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:23:44.889047 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 05:23:44.889047 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 05:23:44.889047 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 05:23:44.889047 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 05:23:44.889047 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 05:23:44.889047 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 05:23:44.889047 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 25 05:23:44.889047 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Jul 25 05:23:44.889047 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 151/297] 10.222.0.33 () {56 vars in 1099 bytes} [Mon Jul 25 05:23:44 2022] GET /identity/v3/roles/reader => generated 84 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:23:44.895002 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a83f39da-8e61-46ed-8a34-2c1d847758ef None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:23:44.991104 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d3d34a-9a5e-40de-a844-e7e9b22eafc8 None None] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:23:44.991104 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d3d34a-9a5e-40de-a844-e7e9b22eafc8 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:23:44.991352 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d3d34a-9a5e-40de-a844-e7e9b22eafc8 None None] PATH_INFO: `/` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:23:44.993042 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 152/298] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jul 25 05:23:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:23:44.999270 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b6b46d-0511-47cb-bbe3-61f62ed8e567 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:23:44.999708 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b6b46d-0511-47cb-bbe3-61f62ed8e567 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:23:45.000061 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b6b46d-0511-47cb-bbe3-61f62ed8e567 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:23:45.011110 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a83f39da-8e61-46ed-8a34-2c1d847758ef None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:23:45.021880 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-c5b6b46d-0511-47cb-bbe3-61f62ed8e567 None None] MFA Rules not processed for user `1eae540242d94c10b0ad33960a855f08`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:23:45.023569 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-c5b6b46d-0511-47cb-bbe3-61f62ed8e567 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:23:45.031522 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:23:45.050812 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ae491a-36f8-4256-8748-db68da22f2c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:23:45.051770 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ae491a-36f8-4256-8748-db68da22f2c7 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:23:45.052225 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ae491a-36f8-4256-8748-db68da22f2c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:23:45.052842 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ae491a-36f8-4256-8748-db68da22f2c7 None admin] PATH_INFO: `/v3/roles` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:23:45.053797 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1ae491a-36f8-4256-8748-db68da22f2c7 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:23:45.054298 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1ae491a-36f8-4256-8748-db68da22f2c7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:23:45.055949 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1ae491a-36f8-4256-8748-db68da22f2c7 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:23:45.066608 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 147/299] 10.222.0.33 () {56 vars in 1101 bytes} [Mon Jul 25 05:23:44 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:23:45.071954 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ae491a-36f8-4256-8748-db68da22f2c7 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:23:45.114522 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 153/300] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jul 25 05:23:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 05:23:45.135899 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e084a25-99a3-47ea-86b3-753044b89119 None None] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:23:45.136193 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e084a25-99a3-47ea-86b3-753044b89119 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:23:45.136464 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e084a25-99a3-47ea-86b3-753044b89119 None None] PATH_INFO: `/` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:23:45.138111 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 154/301] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jul 25 05:23:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:23:45.139550 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e084a25-99a3-47ea-86b3-753044b89119 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:23:45.182458 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ae491a-36f8-4256-8748-db68da22f2c7 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:23:45.219654 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6867e6d9-1fc9-4039-8c9b-cda923c18869 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:23:45.220535 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6867e6d9-1fc9-4039-8c9b-cda923c18869 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:23:45.220847 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6867e6d9-1fc9-4039-8c9b-cda923c18869 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:23:45.221384 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6867e6d9-1fc9-4039-8c9b-cda923c18869 None admin] PATH_INFO: `/v3/users/f0f6a138a10347da9a7ff69583e93b05` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:23:45.236087 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6867e6d9-1fc9-4039-8c9b-cda923c18869 None admin] RBAC: Authorizing `identity:get_user(user_id=f0f6a138a10347da9a7ff69583e93b05)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:23:45.237154 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:23:45.237553 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6867e6d9-1fc9-4039-8c9b-cda923c18869 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:23:45.239329 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 148/302] 10.222.0.33 () {56 vars in 1177 bytes} [Mon Jul 25 05:23:45 2022] GET /identity/v3/users/f0f6a138a10347da9a7ff69583e93b05 => generated 293 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:23:45.240414 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc236ba-6f55-4bd5-925f-c1a951946e16 None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:23:45.241038 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc236ba-6f55-4bd5-925f-c1a951946e16 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:23:45.241241 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc236ba-6f55-4bd5-925f-c1a951946e16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:23:45.250368 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e084a25-99a3-47ea-86b3-753044b89119 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:23:45.258130 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-0dc236ba-6f55-4bd5-925f-c1a951946e16 None None] MFA Rules not processed for user `1eae540242d94c10b0ad33960a855f08`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:23:45.259787 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-0dc236ba-6f55-4bd5-925f-c1a951946e16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:23:45.288441 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50532b67-372f-4b3c-9b5a-6778de9f4948 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:23:45.289566 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50532b67-372f-4b3c-9b5a-6778de9f4948 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:23:45.289855 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50532b67-372f-4b3c-9b5a-6778de9f4948 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:23:45.290135 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50532b67-372f-4b3c-9b5a-6778de9f4948 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:23:45.295544 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:23:45.300912 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50532b67-372f-4b3c-9b5a-6778de9f4948 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 05:23:45.308705 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application [None req-50532b67-372f-4b3c-9b5a-6778de9f4948 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 25 05:23:45.308705 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:23:45.308705 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:23:45.308705 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:23:45.308705 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:23:45.308705 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:23:45.308705 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:23:45.308705 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:23:45.308705 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:23:45.308705 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:23:45.308705 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:23:45.308705 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:23:45.308705 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 05:23:45.308705 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 05:23:45.308705 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 05:23:45.308705 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 05:23:45.309684 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 05:23:45.309684 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 05:23:45.309684 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 05:23:45.309684 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 05:23:45.309684 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:23:45.309684 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:23:45.309684 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:23:45.309684 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:23:45.309684 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:23:45.309684 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:23:45.309684 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:23:45.309684 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application with Lock( Jul 25 05:23:45.309684 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:23:45.309684 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:23:45.309684 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:23:45.309684 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:23:45.310510 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:23:45.310510 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:23:45.310510 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:23:45.310510 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:23:45.310510 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 05:23:45.310510 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 05:23:45.310510 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 05:23:45.310510 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 05:23:45.310510 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 05:23:45.310510 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 05:23:45.310510 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 25 05:23:45.310510 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Jul 25 05:23:45.310510 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 155/303] 10.222.0.33 () {56 vars in 1099 bytes} [Mon Jul 25 05:23:45 2022] GET /identity/v3/roles/member => generated 84 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:23:45.311164 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50532b67-372f-4b3c-9b5a-6778de9f4948 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:23:45.337323 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 149/304] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jul 25 05:23:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 05:23:45.337948 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc236ba-6f55-4bd5-925f-c1a951946e16 None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:23:45.422104 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50532b67-372f-4b3c-9b5a-6778de9f4948 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:23:45.442496 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc236ba-6f55-4bd5-925f-c1a951946e16 None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:23:45.455102 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f6f630-67bd-4c61-a963-b6f9018d53d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:23:45.455771 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f6f630-67bd-4c61-a963-b6f9018d53d7 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:23:45.456062 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f6f630-67bd-4c61-a963-b6f9018d53d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:23:45.456343 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f6f630-67bd-4c61-a963-b6f9018d53d7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:23:45.457222 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55f6f630-67bd-4c61-a963-b6f9018d53d7 None admin] RBAC: Adding query filter params (role.id=be79956018b14baba7a021e2adf27084, user.id=f0f6a138a10347da9a7ff69583e93b05, scope.system=all) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:23:45.457537 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55f6f630-67bd-4c61-a963-b6f9018d53d7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:23:45.458997 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55f6f630-67bd-4c61-a963-b6f9018d53d7 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:23:45.464032 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 156/305] 10.222.0.33 () {56 vars in 1308 bytes} [Mon Jul 25 05:23:45 2022] GET /identity/v3/role_assignments?user.id=f0f6a138a10347da9a7ff69583e93b05&scope.system=all&role.id=be79956018b14baba7a021e2adf27084 => generated 226 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:23:45.465104 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff67190c-103c-41e9-ba0f-ba4855cf7ca1 None None] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:23:45.465396 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff67190c-103c-41e9-ba0f-ba4855cf7ca1 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:23:45.465678 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff67190c-103c-41e9-ba0f-ba4855cf7ca1 None None] PATH_INFO: `/` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:23:45.467641 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 157/306] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jul 25 05:23:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:23:45.468496 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff67190c-103c-41e9-ba0f-ba4855cf7ca1 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:23:45.484044 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f8f7392-2024-4c0c-8faf-7886324b4786 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:23:45.484904 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8f7392-2024-4c0c-8faf-7886324b4786 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:23:45.485191 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8f7392-2024-4c0c-8faf-7886324b4786 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:23:45.485463 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8f7392-2024-4c0c-8faf-7886324b4786 None admin] PATH_INFO: `/v3/roles` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:23:45.486230 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f8f7392-2024-4c0c-8faf-7886324b4786 None admin] RBAC: Adding query filter params (name=member) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:23:45.487122 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f8f7392-2024-4c0c-8faf-7886324b4786 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:23:45.487974 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f8f7392-2024-4c0c-8faf-7886324b4786 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:23:45.497688 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 150/307] 10.222.0.33 () {56 vars in 1101 bytes} [Mon Jul 25 05:23:45 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:23:45.500303 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:23:45.504962 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077dfb57-da86-4ef1-80cb-aaa72e399506 None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:23:45.505399 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077dfb57-da86-4ef1-80cb-aaa72e399506 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:23:45.505699 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077dfb57-da86-4ef1-80cb-aaa72e399506 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:23:45.523867 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-077dfb57-da86-4ef1-80cb-aaa72e399506 None None] MFA Rules not processed for user `1eae540242d94c10b0ad33960a855f08`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:23:45.525201 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-077dfb57-da86-4ef1-80cb-aaa72e399506 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:23:45.554941 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:23:45.596387 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff67190c-103c-41e9-ba0f-ba4855cf7ca1 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:23:45.603846 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 151/308] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jul 25 05:23:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 05:23:45.605335 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-077dfb57-da86-4ef1-80cb-aaa72e399506 None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:23:45.642723 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae84d32-b6df-414b-8908-398f94d41a0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:23:45.643521 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae84d32-b6df-414b-8908-398f94d41a0c None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:23:45.643805 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae84d32-b6df-414b-8908-398f94d41a0c None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:23:45.644256 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae84d32-b6df-414b-8908-398f94d41a0c None admin] PATH_INFO: `/v3/domains/default` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:23:45.645284 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ae84d32-b6df-414b-8908-398f94d41a0c None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:23:45.646714 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ae84d32-b6df-414b-8908-398f94d41a0c None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:23:45.648553 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 158/309] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Jul 25 05:23:45 2022] GET /identity/v3/domains/default => generated 199 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:23:45.649267 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae84d32-b6df-414b-8908-398f94d41a0c None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:23:45.723087 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-077dfb57-da86-4ef1-80cb-aaa72e399506 None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:23:45.742054 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:23:45.758955 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae84d32-b6df-414b-8908-398f94d41a0c None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:23:45.760669 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-606c91ef-2051-4792-9b18-596df8307aea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:23:45.761905 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606c91ef-2051-4792-9b18-596df8307aea None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:23:45.762016 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606c91ef-2051-4792-9b18-596df8307aea None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:23:45.762212 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606c91ef-2051-4792-9b18-596df8307aea None admin] PATH_INFO: `/v3/users/656c60432bfb4d3e95b1250387de6b06` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:23:45.763409 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-606c91ef-2051-4792-9b18-596df8307aea None admin] RBAC: Authorizing `identity:get_user(user_id=656c60432bfb4d3e95b1250387de6b06)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:23:45.765125 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-606c91ef-2051-4792-9b18-596df8307aea None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:23:45.771293 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 152/310] 10.222.0.33 () {56 vars in 1177 bytes} [Mon Jul 25 05:23:45 2022] GET /identity/v3/users/656c60432bfb4d3e95b1250387de6b06 => generated 293 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:23:45.771293 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-606c91ef-2051-4792-9b18-596df8307aea None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:23:45.806056 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a57b83e-2289-4a92-9ed2-d74e00741623 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:23:45.806863 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a57b83e-2289-4a92-9ed2-d74e00741623 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:23:45.807146 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a57b83e-2289-4a92-9ed2-d74e00741623 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:23:45.807243 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:23:45.807421 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a57b83e-2289-4a92-9ed2-d74e00741623 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:23:45.819375 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a57b83e-2289-4a92-9ed2-d74e00741623 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 05:23:45.827317 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application [None req-0a57b83e-2289-4a92-9ed2-d74e00741623 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 25 05:23:45.827317 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:23:45.827317 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:23:45.827317 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:23:45.827317 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:23:45.827317 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:23:45.827317 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:23:45.827317 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:23:45.827317 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:23:45.827317 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:23:45.827317 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:23:45.827317 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:23:45.827317 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 05:23:45.827317 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 05:23:45.827317 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 05:23:45.827317 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 05:23:45.828223 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 05:23:45.828223 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 05:23:45.828223 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 05:23:45.828223 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 05:23:45.828223 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:23:45.828223 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:23:45.828223 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:23:45.828223 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:23:45.828223 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:23:45.828223 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:23:45.828223 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:23:45.828223 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application with Lock( Jul 25 05:23:45.828223 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:23:45.828223 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:23:45.828223 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:23:45.828223 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:23:45.829065 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:23:45.829065 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:23:45.829065 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:23:45.829065 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:23:45.829065 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 05:23:45.829065 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 05:23:45.829065 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 05:23:45.829065 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 05:23:45.829065 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 05:23:45.829065 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 05:23:45.829065 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 25 05:23:45.829065 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Jul 25 05:23:45.829065 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 159/311] 10.222.0.33 () {56 vars in 1099 bytes} [Mon Jul 25 05:23:45 2022] GET /identity/v3/roles/reader => generated 84 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:23:45.829722 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a57b83e-2289-4a92-9ed2-d74e00741623 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:23:45.891230 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-606c91ef-2051-4792-9b18-596df8307aea None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:23:45.928187 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-646a78bc-54bd-493a-be0a-74a956b7ecd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:23:45.928886 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646a78bc-54bd-493a-be0a-74a956b7ecd3 None admin] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:23:45.929179 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646a78bc-54bd-493a-be0a-74a956b7ecd3 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:23:45.929451 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646a78bc-54bd-493a-be0a-74a956b7ecd3 None admin] PATH_INFO: `/v3/groups` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:23:45.930147 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-646a78bc-54bd-493a-be0a-74a956b7ecd3 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:23:45.931578 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-646a78bc-54bd-493a-be0a-74a956b7ecd3 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:23:45.939761 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a57b83e-2289-4a92-9ed2-d74e00741623 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:23:45.956827 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 153/312] 10.222.0.33 () {60 vars in 1134 bytes} [Mon Jul 25 05:23:45 2022] POST /identity/v3/groups => generated 229 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 05:23:45.956827 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-646a78bc-54bd-493a-be0a-74a956b7ecd3 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:23:45.992636 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa5ab527-36ff-4f79-9fa1-610a65127d59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:23:45.993473 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5ab527-36ff-4f79-9fa1-610a65127d59 None admin] REQUEST_METHOD: `PUT` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:23:45.994024 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5ab527-36ff-4f79-9fa1-610a65127d59 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:23:45.994311 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5ab527-36ff-4f79-9fa1-610a65127d59 None admin] PATH_INFO: `/v3/system/users/656c60432bfb4d3e95b1250387de6b06/roles/5074e74e35f54d4b9bb415c34999a82a` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:23:45.996606 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa5ab527-36ff-4f79-9fa1-610a65127d59 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=656c60432bfb4d3e95b1250387de6b06, role_id=5074e74e35f54d4b9bb415c34999a82a)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:23:45.998441 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa5ab527-36ff-4f79-9fa1-610a65127d59 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:23:46.010652 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 160/313] 10.222.0.33 () {58 vars in 1334 bytes} [Mon Jul 25 05:23:45 2022] PUT /identity/v3/system/users/656c60432bfb4d3e95b1250387de6b06/roles/5074e74e35f54d4b9bb415c34999a82a => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 05:23:46.075503 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:23:46.095155 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-646a78bc-54bd-493a-be0a-74a956b7ecd3 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:23:46.143247 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5409d7bf-4e2e-4c2a-a6ab-d4edcdef3dcb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:23:46.143963 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5409d7bf-4e2e-4c2a-a6ab-d4edcdef3dcb None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:23:46.144249 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5409d7bf-4e2e-4c2a-a6ab-d4edcdef3dcb None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:23:46.144532 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5409d7bf-4e2e-4c2a-a6ab-d4edcdef3dcb None admin] PATH_INFO: `/v3/roles` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:23:46.145356 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5409d7bf-4e2e-4c2a-a6ab-d4edcdef3dcb None admin] RBAC: Adding query filter params (name=reader) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:23:46.145673 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5409d7bf-4e2e-4c2a-a6ab-d4edcdef3dcb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:23:46.147173 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5409d7bf-4e2e-4c2a-a6ab-d4edcdef3dcb None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:23:46.159588 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 154/314] 10.222.0.33 () {56 vars in 1101 bytes} [Mon Jul 25 05:23:45 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:23:46.165659 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa5ab527-36ff-4f79-9fa1-610a65127d59 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:23:46.181654 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:23:46.293570 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa5ab527-36ff-4f79-9fa1-610a65127d59 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:23:46.332425 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b2cc7eb-3c8a-4967-884f-6b7e2612619e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:23:46.333415 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2cc7eb-3c8a-4967-884f-6b7e2612619e None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:23:46.333703 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2cc7eb-3c8a-4967-884f-6b7e2612619e None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:23:46.333981 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2cc7eb-3c8a-4967-884f-6b7e2612619e None admin] PATH_INFO: `/v3/users/e03df72adbd34b80951f6d287621b9b9` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:23:46.335231 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b2cc7eb-3c8a-4967-884f-6b7e2612619e None admin] RBAC: Authorizing `identity:get_user(user_id=e03df72adbd34b80951f6d287621b9b9)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:23:46.336957 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b2cc7eb-3c8a-4967-884f-6b7e2612619e None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:23:46.339008 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 161/315] 10.222.0.33 () {56 vars in 1177 bytes} [Mon Jul 25 05:23:46 2022] GET /identity/v3/users/e03df72adbd34b80951f6d287621b9b9 => generated 297 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:23:46.345556 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5409d7bf-4e2e-4c2a-a6ab-d4edcdef3dcb None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:23:46.458521 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5409d7bf-4e2e-4c2a-a6ab-d4edcdef3dcb None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:23:46.496303 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c13459fa-73ae-4c81-a8ee-473a4f6d3a80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:23:46.497248 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13459fa-73ae-4c81-a8ee-473a4f6d3a80 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:23:46.497675 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13459fa-73ae-4c81-a8ee-473a4f6d3a80 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:23:46.498152 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13459fa-73ae-4c81-a8ee-473a4f6d3a80 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:23:46.520513 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:23:46.577868 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c13459fa-73ae-4c81-a8ee-473a4f6d3a80 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:23:46.579268 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c13459fa-73ae-4c81-a8ee-473a4f6d3a80 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:23:46.694517 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 155/316] 10.222.0.33 () {58 vars in 1282 bytes} [Mon Jul 25 05:23:46 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 350 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 25 05:23:46.703104 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b2cc7eb-3c8a-4967-884f-6b7e2612619e None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:23:46.791422 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:23:46.817872 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b2cc7eb-3c8a-4967-884f-6b7e2612619e None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:23:46.853135 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f4ac8a6-f26c-4c0a-ac19-8eac401d3c79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:23:46.854102 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4ac8a6-f26c-4c0a-ac19-8eac401d3c79 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:23:46.854390 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4ac8a6-f26c-4c0a-ac19-8eac401d3c79 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:23:46.854672 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4ac8a6-f26c-4c0a-ac19-8eac401d3c79 None admin] PATH_INFO: `/v3/projects/ec1ed227a23f453cab64e15036af0c60` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:23:46.855905 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f4ac8a6-f26c-4c0a-ac19-8eac401d3c79 None admin] RBAC: Authorizing `identity:get_project(project_id=ec1ed227a23f453cab64e15036af0c60)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:23:46.857948 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f4ac8a6-f26c-4c0a-ac19-8eac401d3c79 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:23:46.860020 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 162/317] 10.222.0.33 () {56 vars in 1186 bytes} [Mon Jul 25 05:23:46 2022] GET /identity/v3/projects/ec1ed227a23f453cab64e15036af0c60 => generated 302 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:23:46.877452 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c13459fa-73ae-4c81-a8ee-473a4f6d3a80 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:23:46.932879 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:23:46.984557 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c13459fa-73ae-4c81-a8ee-473a4f6d3a80 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:23:47.018352 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f423d1-23dd-4605-b04d-cf1308e12cd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:23:47.019279 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f423d1-23dd-4605-b04d-cf1308e12cd5 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:23:47.019785 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f423d1-23dd-4605-b04d-cf1308e12cd5 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:23:47.020280 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f423d1-23dd-4605-b04d-cf1308e12cd5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:23:47.021343 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9f423d1-23dd-4605-b04d-cf1308e12cd5 None admin] RBAC: Adding query filter params (role.id=be79956018b14baba7a021e2adf27084, scope.project.id=ec1ed227a23f453cab64e15036af0c60, user.id=e03df72adbd34b80951f6d287621b9b9) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:23:47.021852 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9f423d1-23dd-4605-b04d-cf1308e12cd5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:23:47.023500 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9f423d1-23dd-4605-b04d-cf1308e12cd5 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:23:47.036457 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 156/318] 10.222.0.33 () {56 vars in 1374 bytes} [Mon Jul 25 05:23:46 2022] GET /identity/v3/role_assignments?user.id=e03df72adbd34b80951f6d287621b9b9&scope.project.id=ec1ed227a23f453cab64e15036af0c60&role.id=be79956018b14baba7a021e2adf27084 => generated 608 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:23:47.696707 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405115b7-50fa-47a4-b584-a53bd3d3b892 None None] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:23:47.697119 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405115b7-50fa-47a4-b584-a53bd3d3b892 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:23:47.697303 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405115b7-50fa-47a4-b584-a53bd3d3b892 None None] PATH_INFO: `/` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:23:47.698722 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 163/319] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jul 25 05:23:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:23:47.704482 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9d2184-c358-4108-85bc-26176fb5250d None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:23:47.704954 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9d2184-c358-4108-85bc-26176fb5250d None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:23:47.705073 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9d2184-c358-4108-85bc-26176fb5250d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:23:47.722164 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-9c9d2184-c358-4108-85bc-26176fb5250d None None] MFA Rules not processed for user `1eae540242d94c10b0ad33960a855f08`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:23:47.723556 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-9c9d2184-c358-4108-85bc-26176fb5250d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:23:47.792682 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 157/320] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jul 25 05:23:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 05:23:47.815955 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-405115b7-50fa-47a4-b584-a53bd3d3b892 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:23:47.922217 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-405115b7-50fa-47a4-b584-a53bd3d3b892 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:23:47.956507 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-589028b9-6ddf-4a2d-a76a-4ae893c62d7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:23:47.957255 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589028b9-6ddf-4a2d-a76a-4ae893c62d7e None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:23:47.957543 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589028b9-6ddf-4a2d-a76a-4ae893c62d7e None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:23:47.957818 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589028b9-6ddf-4a2d-a76a-4ae893c62d7e None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:23:47.968542 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-589028b9-6ddf-4a2d-a76a-4ae893c62d7e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 05:23:47.976304 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application [None req-589028b9-6ddf-4a2d-a76a-4ae893c62d7e None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 25 05:23:47.976304 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:23:47.976304 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:23:47.976304 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:23:47.976304 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:23:47.976304 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:23:47.976304 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:23:47.976304 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:23:47.976304 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:23:47.976304 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:23:47.976304 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:23:47.976304 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:23:47.976304 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 05:23:47.976304 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 05:23:47.976304 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 05:23:47.976304 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 05:23:47.977379 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 05:23:47.977379 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 05:23:47.977379 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 05:23:47.977379 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 05:23:47.977379 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:23:47.977379 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:23:47.977379 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:23:47.977379 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:23:47.977379 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:23:47.977379 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:23:47.977379 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:23:47.977379 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application with Lock( Jul 25 05:23:47.977379 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:23:47.977379 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:23:47.977379 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:23:47.977379 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:23:47.983370 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:23:47.983370 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:23:47.983370 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:23:47.983370 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:23:47.983370 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 05:23:47.983370 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 05:23:47.983370 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 05:23:47.983370 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 05:23:47.983370 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 05:23:47.983370 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 05:23:47.983370 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 25 05:23:47.983370 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Jul 25 05:23:47.983370 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 164/321] 10.222.0.33 () {56 vars in 1099 bytes} [Mon Jul 25 05:23:47 2022] GET /identity/v3/roles/reader => generated 84 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:23:47.985527 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c9d2184-c358-4108-85bc-26176fb5250d None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:23:48.031234 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:23:48.086125 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c9d2184-c358-4108-85bc-26176fb5250d None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:23:48.120660 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc77ede0-3693-47f5-a58d-c9dc26c93e89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:23:48.121364 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc77ede0-3693-47f5-a58d-c9dc26c93e89 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:23:48.121656 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc77ede0-3693-47f5-a58d-c9dc26c93e89 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:23:48.121934 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc77ede0-3693-47f5-a58d-c9dc26c93e89 None admin] PATH_INFO: `/v3/roles` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:23:48.122690 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc77ede0-3693-47f5-a58d-c9dc26c93e89 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:23:48.123032 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc77ede0-3693-47f5-a58d-c9dc26c93e89 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:23:48.124413 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc77ede0-3693-47f5-a58d-c9dc26c93e89 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:23:48.132818 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 158/322] 10.222.0.33 () {56 vars in 1101 bytes} [Mon Jul 25 05:23:47 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:23:48.138262 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-589028b9-6ddf-4a2d-a76a-4ae893c62d7e None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:23:48.148396 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:23:48.187427 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84eccdae-d80a-4d6c-9d9f-785faeee4186 None None] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:23:48.187743 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84eccdae-d80a-4d6c-9d9f-785faeee4186 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:23:48.187998 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84eccdae-d80a-4d6c-9d9f-785faeee4186 None None] PATH_INFO: `/` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:23:48.189828 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 159/323] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jul 25 05:23:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 25 05:23:48.195575 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc8b835-3708-4ef9-978a-d6565969429e None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:23:48.196066 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc8b835-3708-4ef9-978a-d6565969429e None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:23:48.196346 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc8b835-3708-4ef9-978a-d6565969429e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:23:48.213792 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-bcc8b835-3708-4ef9-978a-d6565969429e None None] MFA Rules not processed for user `1eae540242d94c10b0ad33960a855f08`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:23:48.214913 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-bcc8b835-3708-4ef9-978a-d6565969429e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:23:48.245642 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-589028b9-6ddf-4a2d-a76a-4ae893c62d7e None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:23:48.278673 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e79b1c66-495d-4e4e-96b0-84efdfd62ee5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:23:48.280145 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79b1c66-495d-4e4e-96b0-84efdfd62ee5 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:23:48.280436 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79b1c66-495d-4e4e-96b0-84efdfd62ee5 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:23:48.280555 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 160/324] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jul 25 05:23:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 25 05:23:48.280719 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79b1c66-495d-4e4e-96b0-84efdfd62ee5 None admin] PATH_INFO: `/v3/users/f0f6a138a10347da9a7ff69583e93b05` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:23:48.281815 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07cd5a73-1bab-4a5c-aa29-f86c7c59f3d6 None None] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:23:48.281983 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e79b1c66-495d-4e4e-96b0-84efdfd62ee5 None admin] RBAC: Authorizing `identity:get_user(user_id=f0f6a138a10347da9a7ff69583e93b05)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:23:48.282095 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07cd5a73-1bab-4a5c-aa29-f86c7c59f3d6 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:23:48.282373 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07cd5a73-1bab-4a5c-aa29-f86c7c59f3d6 None None] PATH_INFO: `/` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:23:48.283412 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e79b1c66-495d-4e4e-96b0-84efdfd62ee5 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:23:48.283814 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 161/325] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jul 25 05:23:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:23:48.285085 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 165/326] 10.222.0.33 () {56 vars in 1177 bytes} [Mon Jul 25 05:23:48 2022] GET /identity/v3/users/f0f6a138a10347da9a7ff69583e93b05 => generated 293 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:23:48.288697 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c13124-25e1-4ab8-af92-06935e665065 None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:23:48.289019 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c13124-25e1-4ab8-af92-06935e665065 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:23:48.289297 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c13124-25e1-4ab8-af92-06935e665065 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:23:48.296611 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e79b1c66-495d-4e4e-96b0-84efdfd62ee5 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:23:48.305774 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-95c13124-25e1-4ab8-af92-06935e665065 None None] MFA Rules not processed for user `1eae540242d94c10b0ad33960a855f08`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:23:48.306976 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-95c13124-25e1-4ab8-af92-06935e665065 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:23:48.335644 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:23:48.375341 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:23:48.376316 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 162/327] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jul 25 05:23:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 05:23:48.377378 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c13124-25e1-4ab8-af92-06935e665065 None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:23:48.407482 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e79b1c66-495d-4e4e-96b0-84efdfd62ee5 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:23:48.442665 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dfa516e-a05f-4461-81d0-40c43972f106 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:23:48.443518 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dfa516e-a05f-4461-81d0-40c43972f106 None admin] REQUEST_METHOD: `PUT` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:23:48.443809 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dfa516e-a05f-4461-81d0-40c43972f106 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:23:48.444076 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dfa516e-a05f-4461-81d0-40c43972f106 None admin] PATH_INFO: `/v3/system/users/f0f6a138a10347da9a7ff69583e93b05/roles/be79956018b14baba7a021e2adf27084` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:23:48.445570 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dfa516e-a05f-4461-81d0-40c43972f106 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=f0f6a138a10347da9a7ff69583e93b05, role_id=be79956018b14baba7a021e2adf27084)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:23:48.446947 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dfa516e-a05f-4461-81d0-40c43972f106 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:23:48.453822 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 166/328] 10.222.0.33 () {58 vars in 1334 bytes} [Mon Jul 25 05:23:48 2022] PUT /identity/v3/system/users/f0f6a138a10347da9a7ff69583e93b05/roles/be79956018b14baba7a021e2adf27084 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 25 05:23:48.454606 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dfa516e-a05f-4461-81d0-40c43972f106 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:23:48.505227 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c13124-25e1-4ab8-af92-06935e665065 None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:23:48.547141 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c603c5-ccf7-4474-90ad-e50fe6e2042a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:23:48.547964 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c603c5-ccf7-4474-90ad-e50fe6e2042a None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:23:48.548249 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c603c5-ccf7-4474-90ad-e50fe6e2042a None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:23:48.548523 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c603c5-ccf7-4474-90ad-e50fe6e2042a None admin] PATH_INFO: `/v3/domains/default` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:23:48.549788 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9c603c5-ccf7-4474-90ad-e50fe6e2042a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:23:48.551323 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9c603c5-ccf7-4474-90ad-e50fe6e2042a None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:23:48.557824 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 163/329] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Jul 25 05:23:48 2022] GET /identity/v3/domains/default => generated 199 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:23:48.558737 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c603c5-ccf7-4474-90ad-e50fe6e2042a None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:23:48.575793 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dfa516e-a05f-4461-81d0-40c43972f106 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:23:48.596754 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:23:48.617580 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2827fb47-5f75-4a2f-977c-6381840b1349 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:23:48.618391 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2827fb47-5f75-4a2f-977c-6381840b1349 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:23:48.618682 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2827fb47-5f75-4a2f-977c-6381840b1349 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:23:48.618959 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2827fb47-5f75-4a2f-977c-6381840b1349 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:23:48.628155 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:23:48.633379 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2827fb47-5f75-4a2f-977c-6381840b1349 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 05:23:48.642463 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application [None req-2827fb47-5f75-4a2f-977c-6381840b1349 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 25 05:23:48.642463 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:23:48.642463 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:23:48.642463 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:23:48.642463 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:23:48.642463 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:23:48.642463 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:23:48.642463 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:23:48.642463 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:23:48.642463 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:23:48.642463 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:23:48.642463 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:23:48.642463 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 05:23:48.642463 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 05:23:48.642463 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 05:23:48.642463 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 05:23:48.643361 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 05:23:48.643361 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 05:23:48.643361 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 05:23:48.643361 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 05:23:48.643361 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:23:48.643361 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:23:48.643361 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:23:48.643361 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:23:48.643361 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:23:48.643361 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:23:48.643361 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:23:48.643361 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application with Lock( Jul 25 05:23:48.643361 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:23:48.643361 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:23:48.643361 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:23:48.643361 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:23:48.644170 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:23:48.644170 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:23:48.644170 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:23:48.644170 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:23:48.644170 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 05:23:48.644170 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 05:23:48.644170 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 05:23:48.644170 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 05:23:48.644170 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 05:23:48.644170 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 05:23:48.644170 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 25 05:23:48.644170 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Jul 25 05:23:48.647040 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 167/330] 10.222.0.33 () {56 vars in 1099 bytes} [Mon Jul 25 05:23:48 2022] GET /identity/v3/roles/member => generated 84 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:23:48.652477 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2827fb47-5f75-4a2f-977c-6381840b1349 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:23:48.674125 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c603c5-ccf7-4474-90ad-e50fe6e2042a None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:23:48.711128 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d083e28-7e73-47b9-92e2-d2f79ccfd61e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:23:48.711795 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d083e28-7e73-47b9-92e2-d2f79ccfd61e None admin] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:23:48.712078 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d083e28-7e73-47b9-92e2-d2f79ccfd61e None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:23:48.712349 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d083e28-7e73-47b9-92e2-d2f79ccfd61e None admin] PATH_INFO: `/v3/groups` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:23:48.713077 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d083e28-7e73-47b9-92e2-d2f79ccfd61e None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:23:48.714518 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d083e28-7e73-47b9-92e2-d2f79ccfd61e None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:23:48.732898 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 164/331] 10.222.0.33 () {60 vars in 1134 bytes} [Mon Jul 25 05:23:48 2022] POST /identity/v3/groups => generated 226 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 05:23:48.765054 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2827fb47-5f75-4a2f-977c-6381840b1349 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:23:48.811587 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e4edec-94fe-4b98-a2db-0ef3b64bf214 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:23:48.812293 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e4edec-94fe-4b98-a2db-0ef3b64bf214 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:23:48.812673 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e4edec-94fe-4b98-a2db-0ef3b64bf214 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:23:48.812990 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e4edec-94fe-4b98-a2db-0ef3b64bf214 None admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:23:48.813777 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00e4edec-94fe-4b98-a2db-0ef3b64bf214 None admin] RBAC: Adding query filter params (name=member) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:23:48.814094 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00e4edec-94fe-4b98-a2db-0ef3b64bf214 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:23:48.815801 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00e4edec-94fe-4b98-a2db-0ef3b64bf214 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:23:48.830447 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 168/332] 10.222.0.33 () {56 vars in 1101 bytes} [Mon Jul 25 05:23:48 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 05:23:48.831353 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d083e28-7e73-47b9-92e2-d2f79ccfd61e None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:23:48.936705 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:23:48.943126 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d083e28-7e73-47b9-92e2-d2f79ccfd61e None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:23:48.983450 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-172a9368-701e-4559-887f-ac52f83bb8ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:23:48.984483 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172a9368-701e-4559-887f-ac52f83bb8ab None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:23:48.984768 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172a9368-701e-4559-887f-ac52f83bb8ab None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:23:48.985070 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172a9368-701e-4559-887f-ac52f83bb8ab None admin] PATH_INFO: `/v3/users/656c60432bfb4d3e95b1250387de6b06` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:23:48.986319 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-172a9368-701e-4559-887f-ac52f83bb8ab None admin] RBAC: Authorizing `identity:get_user(user_id=656c60432bfb4d3e95b1250387de6b06)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:23:48.987872 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-172a9368-701e-4559-887f-ac52f83bb8ab None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:23:48.989664 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 165/333] 10.222.0.33 () {56 vars in 1177 bytes} [Mon Jul 25 05:23:48 2022] GET /identity/v3/users/656c60432bfb4d3e95b1250387de6b06 => generated 293 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 05:23:48.995385 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e4edec-94fe-4b98-a2db-0ef3b64bf214 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:23:49.060577 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:23:49.107843 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e4edec-94fe-4b98-a2db-0ef3b64bf214 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:23:49.145238 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a4dbb6e-a083-4fe6-b4bf-26994baa1e54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:23:49.146180 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4dbb6e-a083-4fe6-b4bf-26994baa1e54 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:23:49.146608 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4dbb6e-a083-4fe6-b4bf-26994baa1e54 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:23:49.147026 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4dbb6e-a083-4fe6-b4bf-26994baa1e54 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:23:49.148012 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a4dbb6e-a083-4fe6-b4bf-26994baa1e54 None admin] RBAC: Adding query filter params (role.id=5074e74e35f54d4b9bb415c34999a82a, user.id=656c60432bfb4d3e95b1250387de6b06, scope.system=all) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:23:49.148462 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a4dbb6e-a083-4fe6-b4bf-26994baa1e54 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:23:49.150132 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a4dbb6e-a083-4fe6-b4bf-26994baa1e54 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:23:49.164343 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 169/334] 10.222.0.33 () {56 vars in 1308 bytes} [Mon Jul 25 05:23:48 2022] GET /identity/v3/role_assignments?user.id=656c60432bfb4d3e95b1250387de6b06&scope.system=all&role.id=5074e74e35f54d4b9bb415c34999a82a => generated 510 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:23:50.580766 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10152d44-8300-467d-9a0f-728761b1f4d5 None None] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:23:50.581309 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10152d44-8300-467d-9a0f-728761b1f4d5 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:23:50.581386 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10152d44-8300-467d-9a0f-728761b1f4d5 None None] PATH_INFO: `/` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:23:50.582873 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 166/335] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jul 25 05:23:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:23:50.588353 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd60cba-d875-4c42-a46a-f882cddd3848 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:23:50.588760 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd60cba-d875-4c42-a46a-f882cddd3848 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:23:50.588969 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd60cba-d875-4c42-a46a-f882cddd3848 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:23:50.612754 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-bdd60cba-d875-4c42-a46a-f882cddd3848 None None] MFA Rules not processed for user `1eae540242d94c10b0ad33960a855f08`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:23:50.614353 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-bdd60cba-d875-4c42-a46a-f882cddd3848 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:23:50.707379 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 170/336] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jul 25 05:23:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 05:23:50.743304 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10152d44-8300-467d-9a0f-728761b1f4d5 None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:23:50.890163 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10152d44-8300-467d-9a0f-728761b1f4d5 None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:23:50.932299 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c95aab28-4c0c-4019-aad2-d2404e6c3cbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:23:50.933138 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95aab28-4c0c-4019-aad2-d2404e6c3cbb None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:23:50.933427 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95aab28-4c0c-4019-aad2-d2404e6c3cbb None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:23:50.933703 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95aab28-4c0c-4019-aad2-d2404e6c3cbb None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:23:50.950267 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c95aab28-4c0c-4019-aad2-d2404e6c3cbb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 05:23:50.958143 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application [None req-c95aab28-4c0c-4019-aad2-d2404e6c3cbb None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 25 05:23:50.958143 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:23:50.958143 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:23:50.958143 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:23:50.958143 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:23:50.958143 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:23:50.958143 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:23:50.958143 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:23:50.958143 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:23:50.958143 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:23:50.958143 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:23:50.958143 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:23:50.958143 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 05:23:50.958143 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 05:23:50.958143 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 05:23:50.958143 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 05:23:50.959038 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 05:23:50.959038 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 05:23:50.959038 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 05:23:50.959038 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 05:23:50.959038 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:23:50.959038 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:23:50.959038 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:23:50.959038 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:23:50.959038 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:23:50.959038 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:23:50.959038 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:23:50.959038 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application with Lock( Jul 25 05:23:50.959038 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:23:50.959038 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:23:50.959038 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:23:50.959038 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:23:50.959933 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:23:50.959933 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:23:50.959933 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:23:50.959933 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:23:50.959933 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 05:23:50.959933 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 05:23:50.959933 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 05:23:50.959933 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 05:23:50.959933 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 05:23:50.959933 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 05:23:50.959933 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 25 05:23:50.959933 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Jul 25 05:23:50.959933 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 167/337] 10.222.0.33 () {56 vars in 1099 bytes} [Mon Jul 25 05:23:50 2022] GET /identity/v3/roles/reader => generated 84 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:23:50.967677 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd60cba-d875-4c42-a46a-f882cddd3848 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:23:51.058523 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:23:51.137165 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd60cba-d875-4c42-a46a-f882cddd3848 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:23:51.184845 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e27eb8b-321b-4a38-908c-0b3d6f3476a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:23:51.186487 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e27eb8b-321b-4a38-908c-0b3d6f3476a3 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:23:51.186981 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e27eb8b-321b-4a38-908c-0b3d6f3476a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:23:51.187460 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e27eb8b-321b-4a38-908c-0b3d6f3476a3 None admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:23:51.188506 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e27eb8b-321b-4a38-908c-0b3d6f3476a3 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:23:51.189042 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e27eb8b-321b-4a38-908c-0b3d6f3476a3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:23:51.191003 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e27eb8b-321b-4a38-908c-0b3d6f3476a3 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:23:51.203219 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 171/338] 10.222.0.33 () {56 vars in 1101 bytes} [Mon Jul 25 05:23:50 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:23:51.209119 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c95aab28-4c0c-4019-aad2-d2404e6c3cbb None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:23:51.252204 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:23:51.297638 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a542436-2dbb-4c54-afde-d9a9a56fc502 None None] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:23:51.298278 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a542436-2dbb-4c54-afde-d9a9a56fc502 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:23:51.298765 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a542436-2dbb-4c54-afde-d9a9a56fc502 None None] PATH_INFO: `/` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:23:51.300394 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 172/339] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jul 25 05:23:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 25 05:23:51.312070 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0834a61-b53e-4833-b1ad-3a8d2d476abf None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:23:51.312695 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0834a61-b53e-4833-b1ad-3a8d2d476abf None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:23:51.313205 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0834a61-b53e-4833-b1ad-3a8d2d476abf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:23:51.344834 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-a0834a61-b53e-4833-b1ad-3a8d2d476abf None None] MFA Rules not processed for user `1eae540242d94c10b0ad33960a855f08`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:23:51.346404 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-a0834a61-b53e-4833-b1ad-3a8d2d476abf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:23:51.355125 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c95aab28-4c0c-4019-aad2-d2404e6c3cbb None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:23:51.406300 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab557e31-6043-4c0b-9d35-f1097c2aa59f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:23:51.407769 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab557e31-6043-4c0b-9d35-f1097c2aa59f None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:23:51.408060 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab557e31-6043-4c0b-9d35-f1097c2aa59f None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:23:51.408340 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab557e31-6043-4c0b-9d35-f1097c2aa59f None admin] PATH_INFO: `/v3/users/f0f6a138a10347da9a7ff69583e93b05` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:23:51.410216 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab557e31-6043-4c0b-9d35-f1097c2aa59f None admin] RBAC: Authorizing `identity:get_user(user_id=f0f6a138a10347da9a7ff69583e93b05)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:23:51.411824 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab557e31-6043-4c0b-9d35-f1097c2aa59f None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:23:51.413809 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 168/340] 10.222.0.33 () {56 vars in 1177 bytes} [Mon Jul 25 05:23:51 2022] GET /identity/v3/users/f0f6a138a10347da9a7ff69583e93b05 => generated 293 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:23:51.420620 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab557e31-6043-4c0b-9d35-f1097c2aa59f None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:23:51.484265 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 173/341] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jul 25 05:23:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 05:23:51.518516 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0834a61-b53e-4833-b1ad-3a8d2d476abf None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:23:51.559465 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:23:51.574721 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab557e31-6043-4c0b-9d35-f1097c2aa59f None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:23:51.584811 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:23:51.626761 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f849c6ff-7c04-4d39-896f-b4a4a8f82b39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:23:51.627430 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f849c6ff-7c04-4d39-896f-b4a4a8f82b39 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:23:51.627724 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f849c6ff-7c04-4d39-896f-b4a4a8f82b39 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:23:51.628004 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f849c6ff-7c04-4d39-896f-b4a4a8f82b39 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:23:51.628991 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f849c6ff-7c04-4d39-896f-b4a4a8f82b39 None admin] RBAC: Adding query filter params (role.id=be79956018b14baba7a021e2adf27084, user.id=f0f6a138a10347da9a7ff69583e93b05, scope.system=all) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:23:51.629310 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f849c6ff-7c04-4d39-896f-b4a4a8f82b39 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:23:51.630890 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f849c6ff-7c04-4d39-896f-b4a4a8f82b39 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:23:51.648713 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 169/342] 10.222.0.33 () {56 vars in 1308 bytes} [Mon Jul 25 05:23:51 2022] GET /identity/v3/role_assignments?user.id=f0f6a138a10347da9a7ff69583e93b05&scope.system=all&role.id=be79956018b14baba7a021e2adf27084 => generated 510 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:23:51.682561 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0834a61-b53e-4833-b1ad-3a8d2d476abf None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:23:51.732422 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f5d5b5-f215-4849-a648-b8c9f8dedb65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:23:51.733575 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f5d5b5-f215-4849-a648-b8c9f8dedb65 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:23:51.733860 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f5d5b5-f215-4849-a648-b8c9f8dedb65 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:23:51.734184 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f5d5b5-f215-4849-a648-b8c9f8dedb65 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:23:51.751563 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32f5d5b5-f215-4849-a648-b8c9f8dedb65 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 05:23:51.759341 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application [None req-32f5d5b5-f215-4849-a648-b8c9f8dedb65 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 25 05:23:51.759341 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:23:51.759341 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:23:51.759341 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:23:51.759341 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:23:51.759341 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:23:51.759341 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:23:51.759341 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:23:51.759341 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:23:51.759341 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:23:51.759341 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:23:51.759341 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:23:51.759341 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 05:23:51.759341 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 05:23:51.759341 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 05:23:51.759341 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 05:23:51.760307 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 05:23:51.760307 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 05:23:51.760307 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 05:23:51.760307 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 05:23:51.760307 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:23:51.760307 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:23:51.760307 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:23:51.760307 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:23:51.760307 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:23:51.760307 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:23:51.760307 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:23:51.760307 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application with Lock( Jul 25 05:23:51.760307 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:23:51.760307 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:23:51.760307 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:23:51.760307 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:23:51.761146 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:23:51.761146 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:23:51.761146 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:23:51.761146 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:23:51.761146 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 05:23:51.761146 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 05:23:51.761146 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 05:23:51.761146 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 05:23:51.761146 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 05:23:51.761146 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 05:23:51.761146 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 25 05:23:51.761146 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Jul 25 05:23:51.761146 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 174/343] 10.222.0.33 () {56 vars in 1099 bytes} [Mon Jul 25 05:23:51 2022] GET /identity/v3/roles/member => generated 84 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:23:51.766920 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f849c6ff-7c04-4d39-896f-b4a4a8f82b39 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:23:51.907978 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f849c6ff-7c04-4d39-896f-b4a4a8f82b39 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:23:51.913487 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74059f5-d340-476b-a16a-0e309fc85b50 None None] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:23:51.913770 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74059f5-d340-476b-a16a-0e309fc85b50 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:23:51.914049 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74059f5-d340-476b-a16a-0e309fc85b50 None None] PATH_INFO: `/` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:23:51.915438 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 175/344] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jul 25 05:23:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 25 05:23:51.920961 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae3cd34-1961-4bd9-8931-c262366382b0 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:23:51.921539 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae3cd34-1961-4bd9-8931-c262366382b0 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:23:51.922041 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae3cd34-1961-4bd9-8931-c262366382b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:23:51.952430 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a95518-3ad0-492a-adc5-204cac36bf81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:23:51.953482 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a95518-3ad0-492a-adc5-204cac36bf81 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:23:51.953942 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a95518-3ad0-492a-adc5-204cac36bf81 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:23:51.954428 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a95518-3ad0-492a-adc5-204cac36bf81 None admin] PATH_INFO: `/v3/roles` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:23:51.955353 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2a95518-3ad0-492a-adc5-204cac36bf81 None admin] RBAC: Adding query filter params (name=member) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:23:51.955503 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-9ae3cd34-1961-4bd9-8931-c262366382b0 None None] MFA Rules not processed for user `1eae540242d94c10b0ad33960a855f08`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:23:51.955990 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2a95518-3ad0-492a-adc5-204cac36bf81 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:23:51.956945 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-9ae3cd34-1961-4bd9-8931-c262366382b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:23:51.957750 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2a95518-3ad0-492a-adc5-204cac36bf81 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:23:51.980707 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 170/345] 10.222.0.33 () {56 vars in 1101 bytes} [Mon Jul 25 05:23:51 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:23:51.987291 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a95518-3ad0-492a-adc5-204cac36bf81 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:23:52.005236 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:23:52.068618 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 176/346] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jul 25 05:23:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 05:23:52.093728 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03129668-8ae4-4347-b486-b1ffa8d47825 None None] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:23:52.094018 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03129668-8ae4-4347-b486-b1ffa8d47825 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:23:52.094302 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03129668-8ae4-4347-b486-b1ffa8d47825 None None] PATH_INFO: `/` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:23:52.096234 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 177/347] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jul 25 05:23:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 25 05:23:52.101765 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196fc76f-99bb-4501-a180-286eefcd2340 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:23:52.102051 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196fc76f-99bb-4501-a180-286eefcd2340 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:23:52.102526 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196fc76f-99bb-4501-a180-286eefcd2340 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:23:52.105353 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a95518-3ad0-492a-adc5-204cac36bf81 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:23:52.121561 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-196fc76f-99bb-4501-a180-286eefcd2340 None None] MFA Rules not processed for user `1eae540242d94c10b0ad33960a855f08`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:23:52.122850 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-196fc76f-99bb-4501-a180-286eefcd2340 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:23:52.142110 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a2c1fb4-91b7-4315-846a-c495e7472fca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:23:52.143045 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2c1fb4-91b7-4315-846a-c495e7472fca None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:23:52.143502 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2c1fb4-91b7-4315-846a-c495e7472fca None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:23:52.144001 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2c1fb4-91b7-4315-846a-c495e7472fca None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:23:52.151242 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:23:52.210553 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 178/348] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jul 25 05:23:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 05:23:52.211352 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-196fc76f-99bb-4501-a180-286eefcd2340 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:23:52.222879 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a2c1fb4-91b7-4315-846a-c495e7472fca None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:23:52.224723 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a2c1fb4-91b7-4315-846a-c495e7472fca None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:23:52.246371 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:23:52.348859 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-196fc76f-99bb-4501-a180-286eefcd2340 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:23:52.356232 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 171/349] 10.222.0.33 () {58 vars in 1282 bytes} [Mon Jul 25 05:23:51 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 25 05:23:52.357491 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a2c1fb4-91b7-4315-846a-c495e7472fca None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:23:52.396250 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-903f7504-a34e-437f-bb74-925e45e285c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:23:52.397133 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903f7504-a34e-437f-bb74-925e45e285c1 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:23:52.397422 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903f7504-a34e-437f-bb74-925e45e285c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:23:52.397702 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903f7504-a34e-437f-bb74-925e45e285c1 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:23:52.415624 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-903f7504-a34e-437f-bb74-925e45e285c1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 05:23:52.424811 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application [None req-903f7504-a34e-437f-bb74-925e45e285c1 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 25 05:23:52.424811 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:23:52.424811 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:23:52.424811 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:23:52.424811 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:23:52.424811 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:23:52.424811 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:23:52.424811 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:23:52.424811 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:23:52.424811 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:23:52.424811 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:23:52.424811 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:23:52.424811 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 05:23:52.424811 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 05:23:52.424811 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 05:23:52.424811 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 05:23:52.425784 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 05:23:52.425784 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 05:23:52.425784 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 05:23:52.425784 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 05:23:52.425784 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:23:52.425784 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:23:52.425784 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:23:52.425784 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:23:52.425784 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:23:52.425784 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:23:52.425784 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:23:52.425784 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application with Lock( Jul 25 05:23:52.425784 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:23:52.425784 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:23:52.425784 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:23:52.425784 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:23:52.427355 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:23:52.427355 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:23:52.427355 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:23:52.427355 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:23:52.427355 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 05:23:52.427355 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 05:23:52.427355 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 05:23:52.427355 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 05:23:52.427355 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 05:23:52.427355 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 05:23:52.427355 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 25 05:23:52.427355 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Jul 25 05:23:52.427355 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 179/350] 10.222.0.33 () {56 vars in 1114 bytes} [Mon Jul 25 05:23:52 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:23:52.433168 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-903f7504-a34e-437f-bb74-925e45e285c1 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:23:52.460222 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:23:52.481140 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a2c1fb4-91b7-4315-846a-c495e7472fca None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:23:52.513811 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:23:52.528452 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecdbfcc3-89e2-46fb-9ab2-1ae32fc652b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:23:52.529373 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecdbfcc3-89e2-46fb-9ab2-1ae32fc652b9 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:23:52.529657 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecdbfcc3-89e2-46fb-9ab2-1ae32fc652b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:23:52.529976 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecdbfcc3-89e2-46fb-9ab2-1ae32fc652b9 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:23:52.543222 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecdbfcc3-89e2-46fb-9ab2-1ae32fc652b9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 05:23:52.553939 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application [None req-ecdbfcc3-89e2-46fb-9ab2-1ae32fc652b9 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 25 05:23:52.553939 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:23:52.553939 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:23:52.553939 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:23:52.553939 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:23:52.553939 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:23:52.553939 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:23:52.553939 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:23:52.553939 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:23:52.553939 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:23:52.553939 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:23:52.553939 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:23:52.553939 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 05:23:52.553939 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 05:23:52.553939 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 05:23:52.553939 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 05:23:52.555906 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 05:23:52.555906 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 05:23:52.555906 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 05:23:52.555906 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 05:23:52.555906 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:23:52.555906 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:23:52.555906 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:23:52.555906 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:23:52.555906 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:23:52.555906 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:23:52.555906 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:23:52.555906 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application with Lock( Jul 25 05:23:52.555906 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:23:52.555906 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:23:52.555906 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:23:52.555906 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:23:52.558071 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:23:52.558071 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:23:52.558071 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:23:52.558071 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:23:52.558071 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 05:23:52.558071 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 05:23:52.558071 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 05:23:52.558071 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 05:23:52.558071 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 05:23:52.558071 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 05:23:52.558071 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 25 05:23:52.558071 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Jul 25 05:23:52.558071 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 172/351] 10.222.0.33 () {56 vars in 1099 bytes} [Mon Jul 25 05:23:52 2022] GET /identity/v3/roles/member => generated 84 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:23:52.558071 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecdbfcc3-89e2-46fb-9ab2-1ae32fc652b9 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:23:52.565673 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-903f7504-a34e-437f-bb74-925e45e285c1 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:23:52.606733 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b418ed7-106b-4f51-9cd3-22c22ef07e6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:23:52.607625 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b418ed7-106b-4f51-9cd3-22c22ef07e6d None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:23:52.607911 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b418ed7-106b-4f51-9cd3-22c22ef07e6d None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:23:52.608299 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b418ed7-106b-4f51-9cd3-22c22ef07e6d None admin] PATH_INFO: `/v3/projects/4b840bd0540b4512b180fc68fd41cedc` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:23:52.612287 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b418ed7-106b-4f51-9cd3-22c22ef07e6d None admin] RBAC: Authorizing `identity:get_project(project_id=4b840bd0540b4512b180fc68fd41cedc)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:23:52.613822 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b418ed7-106b-4f51-9cd3-22c22ef07e6d None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:23:52.616089 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 180/352] 10.222.0.33 () {56 vars in 1186 bytes} [Mon Jul 25 05:23:52 2022] GET /identity/v3/projects/4b840bd0540b4512b180fc68fd41cedc => generated 298 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:23:52.621011 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b418ed7-106b-4f51-9cd3-22c22ef07e6d None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:23:52.683415 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecdbfcc3-89e2-46fb-9ab2-1ae32fc652b9 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:23:52.731528 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9deb781-5c73-4490-b269-ba1caf53930b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:23:52.733205 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9deb781-5c73-4490-b269-ba1caf53930b None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:23:52.733713 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9deb781-5c73-4490-b269-ba1caf53930b None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:23:52.734184 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9deb781-5c73-4490-b269-ba1caf53930b None admin] PATH_INFO: `/v3/roles` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:23:52.735249 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:23:52.736975 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9deb781-5c73-4490-b269-ba1caf53930b None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:23:52.737563 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9deb781-5c73-4490-b269-ba1caf53930b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:23:52.739723 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9deb781-5c73-4490-b269-ba1caf53930b None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:23:52.749886 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b418ed7-106b-4f51-9cd3-22c22ef07e6d None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:23:52.753351 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 173/353] 10.222.0.33 () {56 vars in 1111 bytes} [Mon Jul 25 05:23:52 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:23:52.754408 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9deb781-5c73-4490-b269-ba1caf53930b None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:23:52.794950 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f95ab668-bf7d-4488-b5d6-5ade84e0a6a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:23:52.795699 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:23:52.796817 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95ab668-bf7d-4488-b5d6-5ade84e0a6a8 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:23:52.797218 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95ab668-bf7d-4488-b5d6-5ade84e0a6a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:23:52.797549 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95ab668-bf7d-4488-b5d6-5ade84e0a6a8 None admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:23:52.800606 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f95ab668-bf7d-4488-b5d6-5ade84e0a6a8 None admin] RBAC: Adding query filter params (name=member) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:23:52.800606 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f95ab668-bf7d-4488-b5d6-5ade84e0a6a8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:23:52.800606 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f95ab668-bf7d-4488-b5d6-5ade84e0a6a8 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:23:52.811853 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 181/354] 10.222.0.33 () {56 vars in 1101 bytes} [Mon Jul 25 05:23:52 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:23:52.812911 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f95ab668-bf7d-4488-b5d6-5ade84e0a6a8 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:23:52.904893 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9deb781-5c73-4490-b269-ba1caf53930b None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:23:52.937307 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f95ab668-bf7d-4488-b5d6-5ade84e0a6a8 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:23:52.942683 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06437221-9e9d-41b0-bedd-f1e875c5ae65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:23:52.943685 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06437221-9e9d-41b0-bedd-f1e875c5ae65 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:23:52.944076 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06437221-9e9d-41b0-bedd-f1e875c5ae65 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:23:52.944455 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06437221-9e9d-41b0-bedd-f1e875c5ae65 None admin] PATH_INFO: `/v3/groups/2673ac268143479dafd556fc6b3bc764` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:23:52.952804 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06437221-9e9d-41b0-bedd-f1e875c5ae65 None admin] RBAC: Authorizing `identity:get_group(group_id=2673ac268143479dafd556fc6b3bc764)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:23:52.954420 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06437221-9e9d-41b0-bedd-f1e875c5ae65 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:23:52.956255 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 174/355] 10.222.0.33 () {56 vars in 1180 bytes} [Mon Jul 25 05:23:52 2022] GET /identity/v3/groups/2673ac268143479dafd556fc6b3bc764 => generated 226 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:23:52.958374 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06437221-9e9d-41b0-bedd-f1e875c5ae65 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:23:52.986892 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b800aaa5-d0f7-4ec2-9a54-2b6b5afac3b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:23:52.987560 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b800aaa5-d0f7-4ec2-9a54-2b6b5afac3b4 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:23:52.987842 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b800aaa5-d0f7-4ec2-9a54-2b6b5afac3b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:23:52.988115 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b800aaa5-d0f7-4ec2-9a54-2b6b5afac3b4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:23:53.014075 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:23:53.068045 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b800aaa5-d0f7-4ec2-9a54-2b6b5afac3b4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:23:53.069560 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b800aaa5-d0f7-4ec2-9a54-2b6b5afac3b4 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:23:53.071981 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:23:53.079220 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06437221-9e9d-41b0-bedd-f1e875c5ae65 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:23:53.123787 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08191ae5-b1f0-4dda-9cbd-a0cf96c27db7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:23:53.124620 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08191ae5-b1f0-4dda-9cbd-a0cf96c27db7 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:23:53.124975 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08191ae5-b1f0-4dda-9cbd-a0cf96c27db7 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:23:53.125258 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08191ae5-b1f0-4dda-9cbd-a0cf96c27db7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:23:53.196017 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 182/356] 10.222.0.33 () {58 vars in 1282 bytes} [Mon Jul 25 05:23:52 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 384 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 25 05:23:53.199371 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b800aaa5-d0f7-4ec2-9a54-2b6b5afac3b4 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:23:53.219926 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08191ae5-b1f0-4dda-9cbd-a0cf96c27db7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:23:53.221742 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08191ae5-b1f0-4dda-9cbd-a0cf96c27db7 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:23:53.257895 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:23:53.318313 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b800aaa5-d0f7-4ec2-9a54-2b6b5afac3b4 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:23:53.349545 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:23:53.361990 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 175/357] 10.222.0.33 () {58 vars in 1282 bytes} [Mon Jul 25 05:23:52 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 405 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 25 05:23:53.363315 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8abd3f-8353-47d8-bed8-24045050c309 None None] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:23:53.363606 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8abd3f-8353-47d8-bed8-24045050c309 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:23:53.363885 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8abd3f-8353-47d8-bed8-24045050c309 None None] PATH_INFO: `/` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:23:53.365228 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 176/358] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jul 25 05:23:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:23:53.365913 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd8abd3f-8353-47d8-bed8-24045050c309 None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:23:53.370769 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-524c7bdf-b762-4806-b410-62c677739f8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:23:53.371456 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524c7bdf-b762-4806-b410-62c677739f8c None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:23:53.373169 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524c7bdf-b762-4806-b410-62c677739f8c None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:23:53.373169 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524c7bdf-b762-4806-b410-62c677739f8c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:23:53.374482 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-524c7bdf-b762-4806-b410-62c677739f8c None admin] RBAC: Adding query filter params (group.id=2673ac268143479dafd556fc6b3bc764, role.id=5074e74e35f54d4b9bb415c34999a82a, scope.project.id=4b840bd0540b4512b180fc68fd41cedc) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:23:53.374482 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-524c7bdf-b762-4806-b410-62c677739f8c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:23:53.375601 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-524c7bdf-b762-4806-b410-62c677739f8c None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:23:53.384824 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 183/359] 10.222.0.33 () {56 vars in 1376 bytes} [Mon Jul 25 05:23:53 2022] GET /identity/v3/role_assignments?group.id=2673ac268143479dafd556fc6b3bc764&scope.project.id=4b840bd0540b4512b180fc68fd41cedc&role.id=5074e74e35f54d4b9bb415c34999a82a => generated 260 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:23:53.386448 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-524c7bdf-b762-4806-b410-62c677739f8c None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:23:53.510945 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd8abd3f-8353-47d8-bed8-24045050c309 None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:23:53.535710 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-524c7bdf-b762-4806-b410-62c677739f8c None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:23:53.553591 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:23:53.570875 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e82c27f5-d85b-469e-8bff-eb469db76e88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:23:53.570875 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82c27f5-d85b-469e-8bff-eb469db76e88 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:23:53.570875 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82c27f5-d85b-469e-8bff-eb469db76e88 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:23:53.570875 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82c27f5-d85b-469e-8bff-eb469db76e88 None admin] PATH_INFO: `/v3/projects/4b840bd0540b4512b180fc68fd41cedc` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:23:53.572318 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e82c27f5-d85b-469e-8bff-eb469db76e88 None admin] RBAC: Authorizing `identity:get_project(project_id=4b840bd0540b4512b180fc68fd41cedc)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:23:53.573636 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e82c27f5-d85b-469e-8bff-eb469db76e88 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:23:53.576162 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 177/360] 10.222.0.33 () {56 vars in 1186 bytes} [Mon Jul 25 05:23:53 2022] GET /identity/v3/projects/4b840bd0540b4512b180fc68fd41cedc => generated 298 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:23:53.577712 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df167a51-5b46-4c3d-ad3d-9acea7604603 None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:23:53.578001 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df167a51-5b46-4c3d-ad3d-9acea7604603 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:23:53.578380 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df167a51-5b46-4c3d-ad3d-9acea7604603 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:23:53.583805 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab17e50f-2b80-4024-94a3-5c99859bf653 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:23:53.584683 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab17e50f-2b80-4024-94a3-5c99859bf653 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:23:53.585091 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab17e50f-2b80-4024-94a3-5c99859bf653 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:23:53.586937 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab17e50f-2b80-4024-94a3-5c99859bf653 None admin] PATH_INFO: `/v3/projects/ec1ed227a23f453cab64e15036af0c60` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:23:53.588233 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab17e50f-2b80-4024-94a3-5c99859bf653 None admin] RBAC: Authorizing `identity:get_project(project_id=ec1ed227a23f453cab64e15036af0c60)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:23:53.590389 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab17e50f-2b80-4024-94a3-5c99859bf653 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:23:53.592105 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 184/361] 10.222.0.33 () {56 vars in 1186 bytes} [Mon Jul 25 05:23:53 2022] GET /identity/v3/projects/ec1ed227a23f453cab64e15036af0c60 => generated 302 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:23:53.596312 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5a5ec8-ab82-4185-a93c-a707c7bef478 None None] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:23:53.596312 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5a5ec8-ab82-4185-a93c-a707c7bef478 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:23:53.596312 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5a5ec8-ab82-4185-a93c-a707c7bef478 None None] PATH_INFO: `/` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:23:53.596312 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 185/362] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jul 25 05:23:53 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:23:53.596312 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc5a5ec8-ab82-4185-a93c-a707c7bef478 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:23:53.605478 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-df167a51-5b46-4c3d-ad3d-9acea7604603 None None] MFA Rules not processed for user `1eae540242d94c10b0ad33960a855f08`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:23:53.606697 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-df167a51-5b46-4c3d-ad3d-9acea7604603 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:23:53.686487 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:23:53.698878 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 178/363] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jul 25 05:23:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 05:23:53.699474 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df167a51-5b46-4c3d-ad3d-9acea7604603 None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:23:53.743857 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc5a5ec8-ab82-4185-a93c-a707c7bef478 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:23:53.790063 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e8fa4a4-952f-4c59-ad2d-7fbc132db017 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:23:53.790851 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8fa4a4-952f-4c59-ad2d-7fbc132db017 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:23:53.791130 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8fa4a4-952f-4c59-ad2d-7fbc132db017 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:23:53.791399 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8fa4a4-952f-4c59-ad2d-7fbc132db017 None admin] PATH_INFO: `/v3/groups/2673ac268143479dafd556fc6b3bc764` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:23:53.796049 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e8fa4a4-952f-4c59-ad2d-7fbc132db017 None admin] RBAC: Authorizing `identity:get_group(group_id=2673ac268143479dafd556fc6b3bc764)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:23:53.796049 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e8fa4a4-952f-4c59-ad2d-7fbc132db017 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:23:53.803511 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 186/364] 10.222.0.33 () {56 vars in 1180 bytes} [Mon Jul 25 05:23:53 2022] GET /identity/v3/groups/2673ac268143479dafd556fc6b3bc764 => generated 226 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:23:53.806179 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e71772f-ce2b-4804-b854-638a8ae71cbe None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:23:53.806477 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e71772f-ce2b-4804-b854-638a8ae71cbe None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:23:53.806768 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e71772f-ce2b-4804-b854-638a8ae71cbe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:23:53.825774 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df167a51-5b46-4c3d-ad3d-9acea7604603 None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:23:53.827404 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-1e71772f-ce2b-4804-b854-638a8ae71cbe None None] MFA Rules not processed for user `1eae540242d94c10b0ad33960a855f08`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:23:53.829171 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-1e71772f-ce2b-4804-b854-638a8ae71cbe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:23:53.881523 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c2df79-2a19-4bbf-a1b8-513111893a6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:23:53.882412 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c2df79-2a19-4bbf-a1b8-513111893a6f None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:23:53.882697 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c2df79-2a19-4bbf-a1b8-513111893a6f None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:23:53.882973 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c2df79-2a19-4bbf-a1b8-513111893a6f None admin] PATH_INFO: `/v3/groups/2673ac268143479dafd556fc6b3bc764` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:23:53.885088 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52c2df79-2a19-4bbf-a1b8-513111893a6f None admin] RBAC: Authorizing `identity:get_group(group_id=2673ac268143479dafd556fc6b3bc764)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:23:53.886551 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52c2df79-2a19-4bbf-a1b8-513111893a6f None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:23:53.888399 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 179/365] 10.222.0.33 () {56 vars in 1180 bytes} [Mon Jul 25 05:23:53 2022] GET /identity/v3/groups/2673ac268143479dafd556fc6b3bc764 => generated 226 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:23:53.889261 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c2df79-2a19-4bbf-a1b8-513111893a6f None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:23:53.894074 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:23:53.922339 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 187/366] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jul 25 05:23:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 05:23:53.923139 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e71772f-ce2b-4804-b854-638a8ae71cbe None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:23:53.965146 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:23:54.020533 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c2df79-2a19-4bbf-a1b8-513111893a6f None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:23:54.060831 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e71772f-ce2b-4804-b854-638a8ae71cbe None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:23:54.071412 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de9d555-b7db-40b8-bfd3-14046a430ef1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:23:54.072238 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de9d555-b7db-40b8-bfd3-14046a430ef1 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:23:54.072888 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de9d555-b7db-40b8-bfd3-14046a430ef1 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:23:54.073199 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de9d555-b7db-40b8-bfd3-14046a430ef1 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:23:54.088034 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2de9d555-b7db-40b8-bfd3-14046a430ef1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 05:23:54.096998 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application [None req-2de9d555-b7db-40b8-bfd3-14046a430ef1 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 25 05:23:54.096998 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:23:54.096998 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:23:54.096998 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:23:54.096998 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:23:54.096998 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:23:54.096998 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:23:54.096998 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:23:54.096998 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:23:54.096998 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:23:54.096998 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:23:54.096998 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:23:54.096998 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 05:23:54.096998 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 05:23:54.096998 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 05:23:54.096998 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 05:23:54.097925 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 05:23:54.097925 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 05:23:54.097925 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 05:23:54.097925 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 05:23:54.097925 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:23:54.097925 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:23:54.097925 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:23:54.097925 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:23:54.097925 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:23:54.097925 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:23:54.097925 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:23:54.097925 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application with Lock( Jul 25 05:23:54.097925 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:23:54.097925 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:23:54.097925 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:23:54.097925 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:23:54.100195 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:23:54.100195 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:23:54.100195 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:23:54.100195 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:23:54.100195 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 05:23:54.100195 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 05:23:54.100195 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 05:23:54.100195 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 05:23:54.100195 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 05:23:54.100195 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 05:23:54.100195 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 25 05:23:54.100195 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Jul 25 05:23:54.100195 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 180/367] 10.222.0.33 () {56 vars in 1114 bytes} [Mon Jul 25 05:23:53 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:23:54.100195 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de9d555-b7db-40b8-bfd3-14046a430ef1 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:23:54.116506 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c4327b-c622-4c45-80e4-079157a77183 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:23:54.117214 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c4327b-c622-4c45-80e4-079157a77183 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:23:54.117506 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c4327b-c622-4c45-80e4-079157a77183 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:23:54.117792 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c4327b-c622-4c45-80e4-079157a77183 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:23:54.118647 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4c4327b-c622-4c45-80e4-079157a77183 None admin] RBAC: Adding query filter params (group.id=2673ac268143479dafd556fc6b3bc764, role.id=02aba1f7cbdf4d6595e18af7fb5727e0, scope.project.id=4b840bd0540b4512b180fc68fd41cedc) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:23:54.118966 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4c4327b-c622-4c45-80e4-079157a77183 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:23:54.120418 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4c4327b-c622-4c45-80e4-079157a77183 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:23:54.127249 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 188/368] 10.222.0.33 () {56 vars in 1376 bytes} [Mon Jul 25 05:23:53 2022] GET /identity/v3/role_assignments?group.id=2673ac268143479dafd556fc6b3bc764&scope.project.id=4b840bd0540b4512b180fc68fd41cedc&role.id=02aba1f7cbdf4d6595e18af7fb5727e0 => generated 260 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:23:54.128294 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c4327b-c622-4c45-80e4-079157a77183 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:23:54.194158 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:23:54.250273 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de9d555-b7db-40b8-bfd3-14046a430ef1 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:23:54.267957 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c4327b-c622-4c45-80e4-079157a77183 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:23:54.291667 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:23:54.306993 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9935b8e5-9bfe-4a41-98c5-dc820fd9c646 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:23:54.307803 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9935b8e5-9bfe-4a41-98c5-dc820fd9c646 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:23:54.308092 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9935b8e5-9bfe-4a41-98c5-dc820fd9c646 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:23:54.308535 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9935b8e5-9bfe-4a41-98c5-dc820fd9c646 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:23:54.311297 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2efb1c97-f242-440e-bbef-268931fbd6ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:23:54.311905 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2efb1c97-f242-440e-bbef-268931fbd6ef None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:23:54.312190 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2efb1c97-f242-440e-bbef-268931fbd6ef None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:23:54.312466 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2efb1c97-f242-440e-bbef-268931fbd6ef None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:23:54.313343 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2efb1c97-f242-440e-bbef-268931fbd6ef None admin] RBAC: Adding query filter params (group.id=2673ac268143479dafd556fc6b3bc764, role.id=5074e74e35f54d4b9bb415c34999a82a, scope.project.id=ec1ed227a23f453cab64e15036af0c60) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:23:54.313666 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2efb1c97-f242-440e-bbef-268931fbd6ef None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:23:54.315385 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2efb1c97-f242-440e-bbef-268931fbd6ef None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:23:54.320521 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9935b8e5-9bfe-4a41-98c5-dc820fd9c646 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 05:23:54.322010 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 181/369] 10.222.0.33 () {56 vars in 1376 bytes} [Mon Jul 25 05:23:54 2022] GET /identity/v3/role_assignments?group.id=2673ac268143479dafd556fc6b3bc764&scope.project.id=ec1ed227a23f453cab64e15036af0c60&role.id=5074e74e35f54d4b9bb415c34999a82a => generated 260 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:23:54.322815 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2efb1c97-f242-440e-bbef-268931fbd6ef None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:23:54.331450 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application [None req-9935b8e5-9bfe-4a41-98c5-dc820fd9c646 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 05:23:54.331450 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:23:54.331450 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:23:54.331450 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:23:54.331450 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:23:54.331450 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:23:54.331450 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:23:54.331450 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:23:54.331450 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:23:54.331450 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:23:54.331450 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:23:54.331450 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:23:54.331450 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 05:23:54.331450 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 05:23:54.331450 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 05:23:54.331450 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 05:23:54.332424 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 05:23:54.332424 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 05:23:54.332424 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 05:23:54.332424 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 05:23:54.332424 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:23:54.332424 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:23:54.332424 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:23:54.332424 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:23:54.332424 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:23:54.332424 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:23:54.332424 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:23:54.332424 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application with Lock( Jul 25 05:23:54.332424 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:23:54.332424 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:23:54.332424 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:23:54.332424 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:23:54.333449 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:23:54.333449 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:23:54.333449 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:23:54.333449 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:23:54.333449 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 05:23:54.333449 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 05:23:54.333449 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 05:23:54.333449 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 05:23:54.333449 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 05:23:54.333449 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 05:23:54.333449 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 05:23:54.333449 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Jul 25 05:23:54.333449 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 189/370] 10.222.0.33 () {56 vars in 1096 bytes} [Mon Jul 25 05:23:54 2022] GET /identity/v3/roles/admin => generated 83 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:23:54.342272 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9935b8e5-9bfe-4a41-98c5-dc820fd9c646 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:23:54.465221 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2efb1c97-f242-440e-bbef-268931fbd6ef None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:23:54.523788 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53205787-9335-4894-87ca-b102f4672ff2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:23:54.525407 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53205787-9335-4894-87ca-b102f4672ff2 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:23:54.526161 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53205787-9335-4894-87ca-b102f4672ff2 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:23:54.526636 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53205787-9335-4894-87ca-b102f4672ff2 None admin] PATH_INFO: `/v3/roles` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:23:54.527870 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53205787-9335-4894-87ca-b102f4672ff2 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:23:54.528407 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53205787-9335-4894-87ca-b102f4672ff2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:23:54.530802 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53205787-9335-4894-87ca-b102f4672ff2 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:23:54.536943 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9935b8e5-9bfe-4a41-98c5-dc820fd9c646 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:23:54.540474 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:23:54.543491 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 182/371] 10.222.0.33 () {56 vars in 1111 bytes} [Mon Jul 25 05:23:54 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:23:54.550605 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53205787-9335-4894-87ca-b102f4672ff2 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:23:54.612043 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f0d72e-77dc-4c33-ba68-f4cbb9eb770c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:23:54.612661 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f0d72e-77dc-4c33-ba68-f4cbb9eb770c None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:23:54.612976 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f0d72e-77dc-4c33-ba68-f4cbb9eb770c None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:23:54.613258 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f0d72e-77dc-4c33-ba68-f4cbb9eb770c None admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:23:54.614023 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37f0d72e-77dc-4c33-ba68-f4cbb9eb770c None admin] RBAC: Adding query filter params (name=admin) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:23:54.614334 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37f0d72e-77dc-4c33-ba68-f4cbb9eb770c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:23:54.615718 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37f0d72e-77dc-4c33-ba68-f4cbb9eb770c None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:23:54.640274 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 190/372] 10.222.0.33 () {56 vars in 1099 bytes} [Mon Jul 25 05:23:54 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 299 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:23:54.649212 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f0d72e-77dc-4c33-ba68-f4cbb9eb770c None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:23:54.651264 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:23:54.754505 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53205787-9335-4894-87ca-b102f4672ff2 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:23:54.828329 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbadc391-80ea-4172-8dfc-caeec1f49c92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:23:54.829028 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbadc391-80ea-4172-8dfc-caeec1f49c92 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:23:54.829315 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbadc391-80ea-4172-8dfc-caeec1f49c92 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:23:54.829596 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbadc391-80ea-4172-8dfc-caeec1f49c92 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:23:54.835698 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f0d72e-77dc-4c33-ba68-f4cbb9eb770c None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:23:54.901309 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d481ddc-8de5-402b-ac30-fc686663fa61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:23:54.901902 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d481ddc-8de5-402b-ac30-fc686663fa61 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:23:54.902191 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d481ddc-8de5-402b-ac30-fc686663fa61 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:23:54.902471 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d481ddc-8de5-402b-ac30-fc686663fa61 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:23:54.955848 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbadc391-80ea-4172-8dfc-caeec1f49c92 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:23:54.957369 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbadc391-80ea-4172-8dfc-caeec1f49c92 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:23:54.965215 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:23:55.029725 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d481ddc-8de5-402b-ac30-fc686663fa61 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:23:55.031191 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d481ddc-8de5-402b-ac30-fc686663fa61 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:23:55.059470 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:23:55.168657 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 183/373] 10.222.0.33 () {58 vars in 1282 bytes} [Mon Jul 25 05:23:54 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 619 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 25 05:23:55.170444 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbadc391-80ea-4172-8dfc-caeec1f49c92 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:23:55.204042 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 191/374] 10.222.0.33 () {58 vars in 1282 bytes} [Mon Jul 25 05:23:54 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 555 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 25 05:23:55.210045 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d481ddc-8de5-402b-ac30-fc686663fa61 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:23:55.282694 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbadc391-80ea-4172-8dfc-caeec1f49c92 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:23:55.323463 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ecda9c-a2d5-4930-83ea-e024e88eaff7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:23:55.324338 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ecda9c-a2d5-4930-83ea-e024e88eaff7 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:23:55.324625 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ecda9c-a2d5-4930-83ea-e024e88eaff7 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:23:55.324941 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ecda9c-a2d5-4930-83ea-e024e88eaff7 None admin] PATH_INFO: `/v3/projects/ec1ed227a23f453cab64e15036af0c60` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:23:55.327210 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1ecda9c-a2d5-4930-83ea-e024e88eaff7 None admin] RBAC: Authorizing `identity:get_project(project_id=ec1ed227a23f453cab64e15036af0c60)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:23:55.328686 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1ecda9c-a2d5-4930-83ea-e024e88eaff7 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:23:55.335589 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 184/375] 10.222.0.33 () {56 vars in 1186 bytes} [Mon Jul 25 05:23:55 2022] GET /identity/v3/projects/ec1ed227a23f453cab64e15036af0c60 => generated 302 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:23:55.337078 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ecda9c-a2d5-4930-83ea-e024e88eaff7 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:23:55.340142 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:23:55.358924 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:23:55.418233 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d481ddc-8de5-402b-ac30-fc686663fa61 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:23:55.485577 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6cb6403-ed17-4133-b625-a0c8e02aa780 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:23:55.486374 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6cb6403-ed17-4133-b625-a0c8e02aa780 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:23:55.486658 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6cb6403-ed17-4133-b625-a0c8e02aa780 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:23:55.486938 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6cb6403-ed17-4133-b625-a0c8e02aa780 None admin] PATH_INFO: `/v3/projects/85f4644d3bc94e89a0061a116652482e` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:23:55.489251 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ecda9c-a2d5-4930-83ea-e024e88eaff7 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:23:55.491347 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6cb6403-ed17-4133-b625-a0c8e02aa780 None admin] RBAC: Authorizing `identity:get_project(project_id=85f4644d3bc94e89a0061a116652482e)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:23:55.492818 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6cb6403-ed17-4133-b625-a0c8e02aa780 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:23:55.494568 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 192/376] 10.222.0.33 () {56 vars in 1186 bytes} [Mon Jul 25 05:23:55 2022] GET /identity/v3/projects/85f4644d3bc94e89a0061a116652482e => generated 344 bytes in 285 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:23:55.501791 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6cb6403-ed17-4133-b625-a0c8e02aa780 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:23:55.554209 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87faee28-7b56-46b4-bdaa-1873f91628b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:23:55.555075 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87faee28-7b56-46b4-bdaa-1873f91628b6 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:23:55.555367 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87faee28-7b56-46b4-bdaa-1873f91628b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:23:55.555648 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87faee28-7b56-46b4-bdaa-1873f91628b6 None admin] PATH_INFO: `/v3/groups/2673ac268143479dafd556fc6b3bc764` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:23:55.556964 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87faee28-7b56-46b4-bdaa-1873f91628b6 None admin] RBAC: Authorizing `identity:get_group(group_id=2673ac268143479dafd556fc6b3bc764)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:23:55.559142 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87faee28-7b56-46b4-bdaa-1873f91628b6 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:23:55.568873 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 185/377] 10.222.0.33 () {56 vars in 1180 bytes} [Mon Jul 25 05:23:55 2022] GET /identity/v3/groups/2673ac268143479dafd556fc6b3bc764 => generated 226 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:23:55.570796 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87faee28-7b56-46b4-bdaa-1873f91628b6 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:23:55.685871 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6cb6403-ed17-4133-b625-a0c8e02aa780 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:23:55.727365 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:23:55.735234 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d75c46ea-2b44-4149-933c-203e4aa10b66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:23:55.736160 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75c46ea-2b44-4149-933c-203e4aa10b66 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:23:55.736392 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75c46ea-2b44-4149-933c-203e4aa10b66 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:23:55.736666 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75c46ea-2b44-4149-933c-203e4aa10b66 None admin] PATH_INFO: `/v3/groups/f6f19f9918714f0093cad025b7a07672` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:23:55.743985 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d75c46ea-2b44-4149-933c-203e4aa10b66 None admin] RBAC: Authorizing `identity:get_group(group_id=f6f19f9918714f0093cad025b7a07672)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:23:55.744510 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:23:55.746843 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d75c46ea-2b44-4149-933c-203e4aa10b66 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:23:55.748995 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 193/378] 10.222.0.33 () {56 vars in 1180 bytes} [Mon Jul 25 05:23:55 2022] GET /identity/v3/groups/f6f19f9918714f0093cad025b7a07672 => generated 229 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:23:55.753164 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87faee28-7b56-46b4-bdaa-1873f91628b6 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:23:55.755409 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d75c46ea-2b44-4149-933c-203e4aa10b66 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:23:55.833258 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76cd0f5e-ce41-4440-b161-3d5073891672 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:23:55.834769 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76cd0f5e-ce41-4440-b161-3d5073891672 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:23:55.835363 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76cd0f5e-ce41-4440-b161-3d5073891672 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:23:55.835843 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76cd0f5e-ce41-4440-b161-3d5073891672 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:23:55.836932 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76cd0f5e-ce41-4440-b161-3d5073891672 None admin] RBAC: Adding query filter params (group.id=2673ac268143479dafd556fc6b3bc764, role.id=02aba1f7cbdf4d6595e18af7fb5727e0, scope.project.id=ec1ed227a23f453cab64e15036af0c60) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:23:55.837472 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76cd0f5e-ce41-4440-b161-3d5073891672 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:23:55.839329 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76cd0f5e-ce41-4440-b161-3d5073891672 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:23:55.855066 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 186/379] 10.222.0.33 () {56 vars in 1376 bytes} [Mon Jul 25 05:23:55 2022] GET /identity/v3/role_assignments?group.id=2673ac268143479dafd556fc6b3bc764&scope.project.id=ec1ed227a23f453cab64e15036af0c60&role.id=02aba1f7cbdf4d6595e18af7fb5727e0 => generated 260 bytes in 284 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 05:23:55.856940 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3591df37-b55c-4eae-b27f-5a9445f35ead None None] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:23:55.857549 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3591df37-b55c-4eae-b27f-5a9445f35ead None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:23:55.858048 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3591df37-b55c-4eae-b27f-5a9445f35ead None None] PATH_INFO: `/` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:23:55.863673 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 187/380] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jul 25 05:23:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:23:55.865531 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be161a32-9ede-4b11-9ee0-de881e776806 None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:23:55.866044 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be161a32-9ede-4b11-9ee0-de881e776806 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:23:55.866533 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be161a32-9ede-4b11-9ee0-de881e776806 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:23:55.894869 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-be161a32-9ede-4b11-9ee0-de881e776806 None None] MFA Rules not processed for user `1eae540242d94c10b0ad33960a855f08`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:23:55.896222 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-be161a32-9ede-4b11-9ee0-de881e776806 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:23:55.933698 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d75c46ea-2b44-4149-933c-203e4aa10b66 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:23:55.980545 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fcf0a80-16dd-401b-8f82-f04788b94afb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:23:55.982726 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fcf0a80-16dd-401b-8f82-f04788b94afb None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:23:55.984845 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fcf0a80-16dd-401b-8f82-f04788b94afb None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:23:55.985121 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fcf0a80-16dd-401b-8f82-f04788b94afb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:23:55.986015 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fcf0a80-16dd-401b-8f82-f04788b94afb None admin] RBAC: Adding query filter params (group.id=f6f19f9918714f0093cad025b7a07672, role.id=39e5c666285a42fe8fc2c3afca6260d1, scope.project.id=85f4644d3bc94e89a0061a116652482e) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:23:55.986388 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fcf0a80-16dd-401b-8f82-f04788b94afb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:23:55.987805 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fcf0a80-16dd-401b-8f82-f04788b94afb None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:23:56.006680 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 194/381] 10.222.0.33 () {56 vars in 1376 bytes} [Mon Jul 25 05:23:55 2022] GET /identity/v3/role_assignments?group.id=f6f19f9918714f0093cad025b7a07672&scope.project.id=85f4644d3bc94e89a0061a116652482e&role.id=39e5c666285a42fe8fc2c3afca6260d1 => generated 260 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:23:56.053864 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 188/382] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jul 25 05:23:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 05:23:56.081066 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fcf0a80-16dd-401b-8f82-f04788b94afb None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:23:56.173031 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:23:56.222702 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fcf0a80-16dd-401b-8f82-f04788b94afb None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:23:56.260359 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da00ca6-7ff2-49b4-a188-17c20b5a11ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:23:56.261575 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da00ca6-7ff2-49b4-a188-17c20b5a11ae None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:23:56.262094 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da00ca6-7ff2-49b4-a188-17c20b5a11ae None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:23:56.262629 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da00ca6-7ff2-49b4-a188-17c20b5a11ae None admin] PATH_INFO: `/v3/roles/member` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:23:56.274562 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6da00ca6-7ff2-49b4-a188-17c20b5a11ae None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 05:23:56.283106 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application [None req-6da00ca6-7ff2-49b4-a188-17c20b5a11ae None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 25 05:23:56.283106 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:23:56.283106 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:23:56.283106 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:23:56.283106 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:23:56.283106 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:23:56.283106 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:23:56.283106 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:23:56.283106 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:23:56.283106 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:23:56.283106 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:23:56.283106 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:23:56.283106 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 05:23:56.283106 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 05:23:56.283106 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 05:23:56.283106 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 05:23:56.284046 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 05:23:56.284046 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 05:23:56.284046 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 05:23:56.284046 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 05:23:56.284046 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:23:56.284046 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:23:56.284046 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:23:56.284046 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:23:56.284046 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:23:56.284046 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:23:56.284046 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:23:56.284046 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application with Lock( Jul 25 05:23:56.284046 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:23:56.284046 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:23:56.284046 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:23:56.284046 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:23:56.287135 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:23:56.287135 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:23:56.287135 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:23:56.287135 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:23:56.287135 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 05:23:56.287135 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 05:23:56.287135 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 05:23:56.287135 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 05:23:56.287135 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 05:23:56.287135 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 05:23:56.287135 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 25 05:23:56.287135 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Jul 25 05:23:56.289710 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 195/383] 10.222.0.33 () {56 vars in 1099 bytes} [Mon Jul 25 05:23:56 2022] GET /identity/v3/roles/member => generated 84 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:23:56.297889 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be161a32-9ede-4b11-9ee0-de881e776806 None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:23:56.431357 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:23:56.461501 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be161a32-9ede-4b11-9ee0-de881e776806 None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:23:56.508872 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e1973a-29a5-4fea-9eb5-1e87ee819fdf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:23:56.509552 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e1973a-29a5-4fea-9eb5-1e87ee819fdf None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:23:56.509836 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e1973a-29a5-4fea-9eb5-1e87ee819fdf None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:23:56.510112 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e1973a-29a5-4fea-9eb5-1e87ee819fdf None admin] PATH_INFO: `/v3/roles` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:23:56.510924 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01e1973a-29a5-4fea-9eb5-1e87ee819fdf None admin] RBAC: Adding query filter params (name=member) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:23:56.511170 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01e1973a-29a5-4fea-9eb5-1e87ee819fdf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:23:56.512544 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01e1973a-29a5-4fea-9eb5-1e87ee819fdf None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:23:56.524615 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 189/384] 10.222.0.33 () {56 vars in 1101 bytes} [Mon Jul 25 05:23:56 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:23:56.530424 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da00ca6-7ff2-49b4-a188-17c20b5a11ae None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:23:56.674744 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da00ca6-7ff2-49b4-a188-17c20b5a11ae None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:23:56.702519 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:23:56.715651 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3328ece7-f41b-4133-887f-e149640a5c00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:23:56.716870 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3328ece7-f41b-4133-887f-e149640a5c00 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:23:56.717422 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3328ece7-f41b-4133-887f-e149640a5c00 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:23:56.717911 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3328ece7-f41b-4133-887f-e149640a5c00 None admin] PATH_INFO: `/v3/groups/2673ac268143479dafd556fc6b3bc764` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:23:56.719365 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3328ece7-f41b-4133-887f-e149640a5c00 None admin] RBAC: Authorizing `identity:get_group(group_id=2673ac268143479dafd556fc6b3bc764)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:23:56.721098 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3328ece7-f41b-4133-887f-e149640a5c00 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:23:56.725667 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 196/385] 10.222.0.33 () {56 vars in 1180 bytes} [Mon Jul 25 05:23:56 2022] GET /identity/v3/groups/2673ac268143479dafd556fc6b3bc764 => generated 226 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:23:56.728731 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e1973a-29a5-4fea-9eb5-1e87ee819fdf None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:23:56.739473 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366398cb-2a30-4b41-bf99-3d591995f218 None None] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:23:56.740116 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366398cb-2a30-4b41-bf99-3d591995f218 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:23:56.740656 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366398cb-2a30-4b41-bf99-3d591995f218 None None] PATH_INFO: `/` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:23:56.742587 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 197/386] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jul 25 05:23:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 25 05:23:56.748437 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4812ed5a-d40c-46f3-9f7e-5a48ee7396db None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:23:56.749123 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4812ed5a-d40c-46f3-9f7e-5a48ee7396db None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:23:56.749630 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4812ed5a-d40c-46f3-9f7e-5a48ee7396db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:23:56.773609 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-4812ed5a-d40c-46f3-9f7e-5a48ee7396db None None] MFA Rules not processed for user `1eae540242d94c10b0ad33960a855f08`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:23:56.774720 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-4812ed5a-d40c-46f3-9f7e-5a48ee7396db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:23:56.905219 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 198/387] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jul 25 05:23:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 25 05:23:56.933817 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4812ed5a-d40c-46f3-9f7e-5a48ee7396db None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:23:56.957489 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e1973a-29a5-4fea-9eb5-1e87ee819fdf None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:23:56.999343 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-333437fa-1041-4f63-9511-185d647d7918 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:23:57.000219 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333437fa-1041-4f63-9511-185d647d7918 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:23:57.000501 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333437fa-1041-4f63-9511-185d647d7918 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:23:57.000772 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333437fa-1041-4f63-9511-185d647d7918 None admin] PATH_INFO: `/v3/projects/4b840bd0540b4512b180fc68fd41cedc` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:23:57.002011 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-333437fa-1041-4f63-9511-185d647d7918 None admin] RBAC: Authorizing `identity:get_project(project_id=4b840bd0540b4512b180fc68fd41cedc)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:23:57.003518 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-333437fa-1041-4f63-9511-185d647d7918 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:23:57.005526 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 190/388] 10.222.0.33 () {56 vars in 1186 bytes} [Mon Jul 25 05:23:56 2022] GET /identity/v3/projects/4b840bd0540b4512b180fc68fd41cedc => generated 298 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:23:57.011186 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-333437fa-1041-4f63-9511-185d647d7918 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:23:57.026568 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:23:57.131930 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4812ed5a-d40c-46f3-9f7e-5a48ee7396db None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:23:57.168772 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:23:57.178819 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-333437fa-1041-4f63-9511-185d647d7918 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:23:57.191856 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6594730f-519a-4c1a-ac8f-ba0b9c271479 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:23:57.193436 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6594730f-519a-4c1a-ac8f-ba0b9c271479 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:23:57.194252 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6594730f-519a-4c1a-ac8f-ba0b9c271479 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:23:57.194744 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6594730f-519a-4c1a-ac8f-ba0b9c271479 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:23:57.210110 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6594730f-519a-4c1a-ac8f-ba0b9c271479 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 05:23:57.221481 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application [None req-6594730f-519a-4c1a-ac8f-ba0b9c271479 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 25 05:23:57.221481 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:23:57.221481 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:23:57.221481 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:23:57.221481 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:23:57.221481 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:23:57.221481 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:23:57.221481 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:23:57.221481 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:23:57.221481 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:23:57.221481 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:23:57.221481 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:23:57.221481 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 05:23:57.221481 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 05:23:57.221481 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 05:23:57.221481 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 05:23:57.222413 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 05:23:57.222413 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 05:23:57.222413 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 05:23:57.222413 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 05:23:57.222413 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:23:57.222413 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:23:57.222413 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:23:57.222413 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:23:57.222413 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:23:57.222413 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:23:57.222413 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:23:57.222413 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application with Lock( Jul 25 05:23:57.222413 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:23:57.222413 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:23:57.222413 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:23:57.222413 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:23:57.223230 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:23:57.223230 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:23:57.223230 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:23:57.223230 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:23:57.223230 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 05:23:57.223230 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 05:23:57.223230 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 05:23:57.223230 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 05:23:57.223230 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 05:23:57.223230 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 05:23:57.223230 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 25 05:23:57.223230 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Jul 25 05:23:57.223851 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9366b3f6-e7d1-4c2d-8935-478e2f7170c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:23:57.223851 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9366b3f6-e7d1-4c2d-8935-478e2f7170c2 None admin] REQUEST_METHOD: `PUT` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:23:57.223851 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9366b3f6-e7d1-4c2d-8935-478e2f7170c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:23:57.223851 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9366b3f6-e7d1-4c2d-8935-478e2f7170c2 None admin] PATH_INFO: `/v3/projects/4b840bd0540b4512b180fc68fd41cedc/groups/2673ac268143479dafd556fc6b3bc764/roles/5074e74e35f54d4b9bb415c34999a82a` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:23:57.225988 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9366b3f6-e7d1-4c2d-8935-478e2f7170c2 None admin] RBAC: Authorizing `identity:create_grant(project_id=4b840bd0540b4512b180fc68fd41cedc, group_id=2673ac268143479dafd556fc6b3bc764, role_id=5074e74e35f54d4b9bb415c34999a82a)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:23:57.226402 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 199/389] 10.222.0.33 () {56 vars in 1114 bytes} [Mon Jul 25 05:23:56 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 293 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:23:57.227736 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a92e26c-2208-4fa5-be68-bd59a63d9ddf None None] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:23:57.227941 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9366b3f6-e7d1-4c2d-8935-478e2f7170c2 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:23:57.228216 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a92e26c-2208-4fa5-be68-bd59a63d9ddf None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:23:57.228595 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a92e26c-2208-4fa5-be68-bd59a63d9ddf None None] PATH_INFO: `/` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:23:57.235628 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 200/390] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jul 25 05:23:57 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:23:57.238117 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a92e26c-2208-4fa5-be68-bd59a63d9ddf None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:23:57.255957 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 191/391] 10.222.0.33 () {58 vars in 1442 bytes} [Mon Jul 25 05:23:57 2022] PUT /identity/v3/projects/4b840bd0540b4512b180fc68fd41cedc/groups/2673ac268143479dafd556fc6b3bc764/roles/5074e74e35f54d4b9bb415c34999a82a => generated 0 bytes in 245 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 25 05:23:57.257698 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb962ce-3166-4f7d-a223-c1cf0237b05a None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:23:57.258218 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb962ce-3166-4f7d-a223-c1cf0237b05a None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:23:57.265426 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb962ce-3166-4f7d-a223-c1cf0237b05a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:23:57.291856 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-eeb962ce-3166-4f7d-a223-c1cf0237b05a None None] MFA Rules not processed for user `1eae540242d94c10b0ad33960a855f08`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:23:57.293669 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-eeb962ce-3166-4f7d-a223-c1cf0237b05a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:23:57.373376 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:23:57.379937 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 192/392] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jul 25 05:23:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 05:23:57.421611 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a92e26c-2208-4fa5-be68-bd59a63d9ddf None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:23:57.434303 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb962ce-3166-4f7d-a223-c1cf0237b05a None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:23:57.473024 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b33b4a0-e4cf-4d1b-bc41-f942ef444932 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:23:57.473981 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b33b4a0-e4cf-4d1b-bc41-f942ef444932 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:23:57.474275 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b33b4a0-e4cf-4d1b-bc41-f942ef444932 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:23:57.474559 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b33b4a0-e4cf-4d1b-bc41-f942ef444932 None admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:23:57.475347 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b33b4a0-e4cf-4d1b-bc41-f942ef444932 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:23:57.475664 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b33b4a0-e4cf-4d1b-bc41-f942ef444932 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:23:57.477357 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b33b4a0-e4cf-4d1b-bc41-f942ef444932 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:23:57.495407 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 201/393] 10.222.0.33 () {56 vars in 1111 bytes} [Mon Jul 25 05:23:57 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:23:57.497889 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b33b4a0-e4cf-4d1b-bc41-f942ef444932 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:23:57.557177 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:23:57.576221 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb962ce-3166-4f7d-a223-c1cf0237b05a None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:23:57.623047 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1873fe0-5cc4-4e50-beae-05a09a578308 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:23:57.623946 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1873fe0-5cc4-4e50-beae-05a09a578308 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:23:57.624242 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1873fe0-5cc4-4e50-beae-05a09a578308 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:23:57.624526 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1873fe0-5cc4-4e50-beae-05a09a578308 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:23:57.625782 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b33b4a0-e4cf-4d1b-bc41-f942ef444932 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:23:57.638208 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1873fe0-5cc4-4e50-beae-05a09a578308 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 05:23:57.648569 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application [None req-f1873fe0-5cc4-4e50-beae-05a09a578308 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 25 05:23:57.648569 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:23:57.648569 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:23:57.648569 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:23:57.648569 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:23:57.648569 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:23:57.648569 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:23:57.648569 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:23:57.648569 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:23:57.648569 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:23:57.648569 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:23:57.648569 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:23:57.648569 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 05:23:57.648569 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 05:23:57.648569 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 05:23:57.648569 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 05:23:57.649497 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 05:23:57.649497 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 05:23:57.649497 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 05:23:57.649497 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 05:23:57.649497 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:23:57.649497 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:23:57.649497 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:23:57.649497 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:23:57.649497 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:23:57.649497 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:23:57.649497 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:23:57.649497 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application with Lock( Jul 25 05:23:57.649497 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:23:57.649497 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:23:57.649497 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:23:57.649497 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:23:57.650272 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:23:57.650272 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:23:57.650272 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:23:57.650272 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:23:57.650272 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 05:23:57.650272 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 05:23:57.650272 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 05:23:57.650272 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 05:23:57.650272 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 05:23:57.650272 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 05:23:57.650272 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 25 05:23:57.650272 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Jul 25 05:23:57.652872 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 193/394] 10.222.0.33 () {56 vars in 1099 bytes} [Mon Jul 25 05:23:57 2022] GET /identity/v3/roles/member => generated 84 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 25 05:23:57.658542 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1873fe0-5cc4-4e50-beae-05a09a578308 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:23:57.671330 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdee05e0-c5c4-4dda-b710-72322de3fb76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:23:57.672591 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdee05e0-c5c4-4dda-b710-72322de3fb76 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:23:57.673149 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdee05e0-c5c4-4dda-b710-72322de3fb76 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:23:57.673641 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdee05e0-c5c4-4dda-b710-72322de3fb76 None admin] PATH_INFO: `/v3/groups/2673ac268143479dafd556fc6b3bc764` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:23:57.675118 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdee05e0-c5c4-4dda-b710-72322de3fb76 None admin] RBAC: Authorizing `identity:get_group(group_id=2673ac268143479dafd556fc6b3bc764)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:23:57.676934 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdee05e0-c5c4-4dda-b710-72322de3fb76 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:23:57.679153 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 202/395] 10.222.0.33 () {56 vars in 1180 bytes} [Mon Jul 25 05:23:57 2022] GET /identity/v3/groups/2673ac268143479dafd556fc6b3bc764 => generated 226 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:23:57.685230 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdee05e0-c5c4-4dda-b710-72322de3fb76 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:23:57.735356 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:23:57.793344 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1873fe0-5cc4-4e50-beae-05a09a578308 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:23:57.824848 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdee05e0-c5c4-4dda-b710-72322de3fb76 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:23:57.839040 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:23:57.839995 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-270cbe7a-273c-4842-83e2-968980870d75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:23:57.840581 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270cbe7a-273c-4842-83e2-968980870d75 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:23:57.840906 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270cbe7a-273c-4842-83e2-968980870d75 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:23:57.841767 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270cbe7a-273c-4842-83e2-968980870d75 None admin] PATH_INFO: `/v3/roles` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:23:57.843270 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-270cbe7a-273c-4842-83e2-968980870d75 None admin] RBAC: Adding query filter params (name=member) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:23:57.843661 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-270cbe7a-273c-4842-83e2-968980870d75 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:23:57.846084 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-270cbe7a-273c-4842-83e2-968980870d75 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:23:57.861346 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 194/396] 10.222.0.33 () {56 vars in 1101 bytes} [Mon Jul 25 05:23:57 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:23:57.863731 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-270cbe7a-273c-4842-83e2-968980870d75 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:23:57.885936 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdbdbf73-7ec7-4f30-a8db-0327d912e653 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:23:57.887256 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdbdbf73-7ec7-4f30-a8db-0327d912e653 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:23:57.888317 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdbdbf73-7ec7-4f30-a8db-0327d912e653 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:23:57.888832 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdbdbf73-7ec7-4f30-a8db-0327d912e653 None admin] PATH_INFO: `/v3/projects/4b840bd0540b4512b180fc68fd41cedc` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:23:57.892371 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdbdbf73-7ec7-4f30-a8db-0327d912e653 None admin] RBAC: Authorizing `identity:get_project(project_id=4b840bd0540b4512b180fc68fd41cedc)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:23:57.894238 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdbdbf73-7ec7-4f30-a8db-0327d912e653 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:23:57.896572 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 203/397] 10.222.0.33 () {56 vars in 1186 bytes} [Mon Jul 25 05:23:57 2022] GET /identity/v3/projects/4b840bd0540b4512b180fc68fd41cedc => generated 298 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:23:57.905433 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdbdbf73-7ec7-4f30-a8db-0327d912e653 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:23:57.987089 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-270cbe7a-273c-4842-83e2-968980870d75 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:23:58.015484 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:23:58.033576 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d5aa3e1-6bea-4075-81ba-2eff47c3663a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:23:58.034446 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5aa3e1-6bea-4075-81ba-2eff47c3663a None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:23:58.034733 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5aa3e1-6bea-4075-81ba-2eff47c3663a None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:23:58.035016 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5aa3e1-6bea-4075-81ba-2eff47c3663a None admin] PATH_INFO: `/v3/groups/2673ac268143479dafd556fc6b3bc764` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:23:58.036235 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d5aa3e1-6bea-4075-81ba-2eff47c3663a None admin] RBAC: Authorizing `identity:get_group(group_id=2673ac268143479dafd556fc6b3bc764)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:23:58.038010 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d5aa3e1-6bea-4075-81ba-2eff47c3663a None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:23:58.045040 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 195/398] 10.222.0.33 () {56 vars in 1180 bytes} [Mon Jul 25 05:23:57 2022] GET /identity/v3/groups/2673ac268143479dafd556fc6b3bc764 => generated 226 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 05:23:58.046937 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d5aa3e1-6bea-4075-81ba-2eff47c3663a None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:23:58.110879 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdbdbf73-7ec7-4f30-a8db-0327d912e653 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:23:58.167824 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d5aa3e1-6bea-4075-81ba-2eff47c3663a None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:23:58.186136 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f24bfc4-509e-416c-9d7c-cbcbee2b2ab7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:23:58.187377 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f24bfc4-509e-416c-9d7c-cbcbee2b2ab7 None admin] REQUEST_METHOD: `PUT` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:23:58.187951 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f24bfc4-509e-416c-9d7c-cbcbee2b2ab7 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:23:58.188467 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f24bfc4-509e-416c-9d7c-cbcbee2b2ab7 None admin] PATH_INFO: `/v3/projects/4b840bd0540b4512b180fc68fd41cedc/groups/2673ac268143479dafd556fc6b3bc764/roles/02aba1f7cbdf4d6595e18af7fb5727e0` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:23:58.195951 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f24bfc4-509e-416c-9d7c-cbcbee2b2ab7 None admin] RBAC: Authorizing `identity:create_grant(project_id=4b840bd0540b4512b180fc68fd41cedc, group_id=2673ac268143479dafd556fc6b3bc764, role_id=02aba1f7cbdf4d6595e18af7fb5727e0)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:23:58.197741 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f24bfc4-509e-416c-9d7c-cbcbee2b2ab7 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:23:58.207313 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-215cd8d1-5336-4384-ac5e-81d69b70d011 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:23:58.208165 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215cd8d1-5336-4384-ac5e-81d69b70d011 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:23:58.209297 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215cd8d1-5336-4384-ac5e-81d69b70d011 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:23:58.209710 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215cd8d1-5336-4384-ac5e-81d69b70d011 None admin] PATH_INFO: `/v3/projects/ec1ed227a23f453cab64e15036af0c60` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:23:58.211398 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-215cd8d1-5336-4384-ac5e-81d69b70d011 None admin] RBAC: Authorizing `identity:get_project(project_id=ec1ed227a23f453cab64e15036af0c60)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:23:58.213146 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-215cd8d1-5336-4384-ac5e-81d69b70d011 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:23:58.220461 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 196/399] 10.222.0.33 () {56 vars in 1186 bytes} [Mon Jul 25 05:23:58 2022] GET /identity/v3/projects/ec1ed227a23f453cab64e15036af0c60 => generated 302 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 05:23:58.221694 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-215cd8d1-5336-4384-ac5e-81d69b70d011 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:23:58.237591 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 204/400] 10.222.0.33 () {58 vars in 1442 bytes} [Mon Jul 25 05:23:57 2022] PUT /identity/v3/projects/4b840bd0540b4512b180fc68fd41cedc/groups/2673ac268143479dafd556fc6b3bc764/roles/02aba1f7cbdf4d6595e18af7fb5727e0 => generated 0 bytes in 336 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 25 05:23:58.289576 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:23:58.342077 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-215cd8d1-5336-4384-ac5e-81d69b70d011 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:23:58.391687 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df41fee9-b11f-4fc5-a337-a09f6feac0a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:23:58.392504 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df41fee9-b11f-4fc5-a337-a09f6feac0a0 None admin] REQUEST_METHOD: `PUT` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:23:58.392809 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df41fee9-b11f-4fc5-a337-a09f6feac0a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:23:58.393160 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df41fee9-b11f-4fc5-a337-a09f6feac0a0 None admin] PATH_INFO: `/v3/projects/ec1ed227a23f453cab64e15036af0c60/groups/2673ac268143479dafd556fc6b3bc764/roles/5074e74e35f54d4b9bb415c34999a82a` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:23:58.398329 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df41fee9-b11f-4fc5-a337-a09f6feac0a0 None admin] RBAC: Authorizing `identity:create_grant(project_id=ec1ed227a23f453cab64e15036af0c60, group_id=2673ac268143479dafd556fc6b3bc764, role_id=5074e74e35f54d4b9bb415c34999a82a)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:23:58.399859 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df41fee9-b11f-4fc5-a337-a09f6feac0a0 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:23:58.418540 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 197/401] 10.222.0.33 () {58 vars in 1442 bytes} [Mon Jul 25 05:23:58 2022] PUT /identity/v3/projects/ec1ed227a23f453cab64e15036af0c60/groups/2673ac268143479dafd556fc6b3bc764/roles/5074e74e35f54d4b9bb415c34999a82a => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 25 05:23:58.431989 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f9fa39-cc92-43dd-8421-bb57395ddfce None None] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:23:58.432647 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f9fa39-cc92-43dd-8421-bb57395ddfce None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:23:58.433161 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f9fa39-cc92-43dd-8421-bb57395ddfce None None] PATH_INFO: `/` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:23:58.435771 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 205/402] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jul 25 05:23:58 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:23:58.441694 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7fea519-9598-4872-b1a0-474edf506d78 None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:23:58.442004 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7fea519-9598-4872-b1a0-474edf506d78 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:23:58.442256 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7fea519-9598-4872-b1a0-474edf506d78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:23:58.482226 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-a7fea519-9598-4872-b1a0-474edf506d78 None None] MFA Rules not processed for user `1eae540242d94c10b0ad33960a855f08`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:23:58.483507 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-a7fea519-9598-4872-b1a0-474edf506d78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:23:58.577289 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 198/403] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jul 25 05:23:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 05:23:58.602675 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f9fa39-cc92-43dd-8421-bb57395ddfce None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:23:58.649462 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5230179-1e5f-4240-a652-008d3fd82288 None None] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:23:58.650068 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5230179-1e5f-4240-a652-008d3fd82288 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:23:58.650349 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5230179-1e5f-4240-a652-008d3fd82288 None None] PATH_INFO: `/` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:23:58.651754 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 199/404] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jul 25 05:23:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 25 05:23:58.652563 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:23:58.658846 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d891aec2-6f6a-4995-8408-9ba8063a9655 None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:23:58.659141 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d891aec2-6f6a-4995-8408-9ba8063a9655 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:23:58.659428 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d891aec2-6f6a-4995-8408-9ba8063a9655 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:23:58.685773 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-d891aec2-6f6a-4995-8408-9ba8063a9655 None None] MFA Rules not processed for user `1eae540242d94c10b0ad33960a855f08`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:23:58.688037 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-d891aec2-6f6a-4995-8408-9ba8063a9655 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:23:58.766918 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f9fa39-cc92-43dd-8421-bb57395ddfce None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:23:58.772007 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:23:58.835430 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 200/405] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jul 25 05:23:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 05:23:58.847681 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4cbddb6-fed9-4016-9b00-1a98b0c19567 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:23:58.848502 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4cbddb6-fed9-4016-9b00-1a98b0c19567 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:23:58.848804 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4cbddb6-fed9-4016-9b00-1a98b0c19567 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:23:58.849078 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4cbddb6-fed9-4016-9b00-1a98b0c19567 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:23:58.859197 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d891aec2-6f6a-4995-8408-9ba8063a9655 None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:23:58.864373 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4cbddb6-fed9-4016-9b00-1a98b0c19567 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 05:23:58.876824 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application [None req-c4cbddb6-fed9-4016-9b00-1a98b0c19567 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 25 05:23:58.876824 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:23:58.876824 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:23:58.876824 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:23:58.876824 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:23:58.876824 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:23:58.876824 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:23:58.876824 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:23:58.876824 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:23:58.876824 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:23:58.876824 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:23:58.876824 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:23:58.876824 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 05:23:58.876824 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 05:23:58.876824 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 05:23:58.876824 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 05:23:58.877753 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 05:23:58.877753 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 05:23:58.877753 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 05:23:58.877753 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 05:23:58.877753 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:23:58.877753 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:23:58.877753 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:23:58.877753 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:23:58.877753 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:23:58.877753 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:23:58.877753 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:23:58.877753 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application with Lock( Jul 25 05:23:58.877753 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:23:58.877753 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:23:58.877753 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:23:58.877753 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:23:58.878548 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:23:58.878548 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:23:58.878548 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:23:58.878548 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:23:58.878548 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 05:23:58.878548 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 05:23:58.878548 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 05:23:58.878548 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 05:23:58.878548 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 05:23:58.878548 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 05:23:58.878548 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 25 05:23:58.878548 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Jul 25 05:23:58.878548 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 206/406] 10.222.0.33 () {56 vars in 1114 bytes} [Mon Jul 25 05:23:58 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 273 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:23:58.881345 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4cbddb6-fed9-4016-9b00-1a98b0c19567 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:23:58.997726 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d891aec2-6f6a-4995-8408-9ba8063a9655 None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:23:59.036104 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4794ed9f-81e0-4d7f-8704-b839644c5ebc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:23:59.037341 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4794ed9f-81e0-4d7f-8704-b839644c5ebc None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:23:59.037858 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4794ed9f-81e0-4d7f-8704-b839644c5ebc None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:23:59.038337 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4794ed9f-81e0-4d7f-8704-b839644c5ebc None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:23:59.051302 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4794ed9f-81e0-4d7f-8704-b839644c5ebc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 05:23:59.060712 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application [None req-4794ed9f-81e0-4d7f-8704-b839644c5ebc None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 05:23:59.060712 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:23:59.060712 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:23:59.060712 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:23:59.060712 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:23:59.060712 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:23:59.060712 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:23:59.060712 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:23:59.060712 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:23:59.060712 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:23:59.060712 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:23:59.060712 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:23:59.060712 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 05:23:59.060712 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 05:23:59.060712 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 05:23:59.060712 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 05:23:59.061712 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 05:23:59.061712 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 05:23:59.061712 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 05:23:59.061712 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 05:23:59.061712 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:23:59.061712 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:23:59.061712 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:23:59.061712 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:23:59.061712 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:23:59.061712 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:23:59.061712 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:23:59.061712 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application with Lock( Jul 25 05:23:59.061712 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:23:59.061712 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:23:59.061712 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:23:59.061712 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:23:59.062505 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:23:59.062505 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:23:59.062505 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:23:59.062505 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:23:59.062505 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 05:23:59.062505 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 05:23:59.062505 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 05:23:59.062505 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 05:23:59.062505 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 05:23:59.062505 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 05:23:59.062505 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 05:23:59.062505 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Jul 25 05:23:59.065614 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 201/407] 10.222.0.33 () {56 vars in 1096 bytes} [Mon Jul 25 05:23:58 2022] GET /identity/v3/roles/admin => generated 83 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 25 05:23:59.071782 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4794ed9f-81e0-4d7f-8704-b839644c5ebc None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:23:59.077144 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:23:59.080014 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4cbddb6-fed9-4016-9b00-1a98b0c19567 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:23:59.115102 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c1ad660-dcf4-4620-bbaf-aa3e664bca5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:23:59.115775 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1ad660-dcf4-4620-bbaf-aa3e664bca5c None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:23:59.116046 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1ad660-dcf4-4620-bbaf-aa3e664bca5c None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:23:59.116351 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1ad660-dcf4-4620-bbaf-aa3e664bca5c None admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:23:59.117106 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c1ad660-dcf4-4620-bbaf-aa3e664bca5c None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:23:59.117415 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c1ad660-dcf4-4620-bbaf-aa3e664bca5c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:23:59.120506 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c1ad660-dcf4-4620-bbaf-aa3e664bca5c None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:23:59.128641 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 207/408] 10.222.0.33 () {56 vars in 1111 bytes} [Mon Jul 25 05:23:58 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:23:59.134039 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c1ad660-dcf4-4620-bbaf-aa3e664bca5c None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:23:59.171214 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:23:59.234373 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c1ad660-dcf4-4620-bbaf-aa3e664bca5c None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:23:59.268738 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d34af5-0787-4fe4-b87b-d439fdbcd449 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:23:59.269545 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d34af5-0787-4fe4-b87b-d439fdbcd449 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:23:59.269828 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d34af5-0787-4fe4-b87b-d439fdbcd449 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:23:59.270104 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d34af5-0787-4fe4-b87b-d439fdbcd449 None admin] PATH_INFO: `/v3/groups/2673ac268143479dafd556fc6b3bc764` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:23:59.271286 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77d34af5-0787-4fe4-b87b-d439fdbcd449 None admin] RBAC: Authorizing `identity:get_group(group_id=2673ac268143479dafd556fc6b3bc764)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:23:59.272755 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77d34af5-0787-4fe4-b87b-d439fdbcd449 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:23:59.274719 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 208/409] 10.222.0.33 () {56 vars in 1180 bytes} [Mon Jul 25 05:23:59 2022] GET /identity/v3/groups/2673ac268143479dafd556fc6b3bc764 => generated 226 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:23:59.280575 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d34af5-0787-4fe4-b87b-d439fdbcd449 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:23:59.296093 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4794ed9f-81e0-4d7f-8704-b839644c5ebc None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:23:59.331183 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d10edd13-f67b-4eb7-aa7f-83bfb2de88cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:23:59.331965 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10edd13-f67b-4eb7-aa7f-83bfb2de88cc None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:23:59.332260 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10edd13-f67b-4eb7-aa7f-83bfb2de88cc None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:23:59.332547 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10edd13-f67b-4eb7-aa7f-83bfb2de88cc None admin] PATH_INFO: `/v3/roles` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:23:59.333364 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d10edd13-f67b-4eb7-aa7f-83bfb2de88cc None admin] RBAC: Adding query filter params (name=admin) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:23:59.333682 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d10edd13-f67b-4eb7-aa7f-83bfb2de88cc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:23:59.335078 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d10edd13-f67b-4eb7-aa7f-83bfb2de88cc None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:23:59.337188 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:23:59.344588 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 202/410] 10.222.0.33 () {56 vars in 1099 bytes} [Mon Jul 25 05:23:59 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 05:23:59.350052 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d10edd13-f67b-4eb7-aa7f-83bfb2de88cc None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:23:59.459553 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d10edd13-f67b-4eb7-aa7f-83bfb2de88cc None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:23:59.467680 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:23:59.495997 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0daf1560-6fbd-4ee6-a6a9-6065cbcfe90a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:23:59.496895 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0daf1560-6fbd-4ee6-a6a9-6065cbcfe90a None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:23:59.497226 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0daf1560-6fbd-4ee6-a6a9-6065cbcfe90a None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:23:59.497516 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0daf1560-6fbd-4ee6-a6a9-6065cbcfe90a None admin] PATH_INFO: `/v3/groups/f6f19f9918714f0093cad025b7a07672` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:23:59.498820 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0daf1560-6fbd-4ee6-a6a9-6065cbcfe90a None admin] RBAC: Authorizing `identity:get_group(group_id=f6f19f9918714f0093cad025b7a07672)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:23:59.500707 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0daf1560-6fbd-4ee6-a6a9-6065cbcfe90a None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:23:59.502678 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 203/411] 10.222.0.33 () {56 vars in 1180 bytes} [Mon Jul 25 05:23:59 2022] GET /identity/v3/groups/f6f19f9918714f0093cad025b7a07672 => generated 229 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:23:59.507940 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0daf1560-6fbd-4ee6-a6a9-6065cbcfe90a None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:23:59.566700 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d34af5-0787-4fe4-b87b-d439fdbcd449 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:23:59.616202 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0daf1560-6fbd-4ee6-a6a9-6065cbcfe90a None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:23:59.652601 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f876b4-d66f-49cf-ba48-133a111b1303 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:23:59.653561 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f876b4-d66f-49cf-ba48-133a111b1303 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:23:59.653852 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f876b4-d66f-49cf-ba48-133a111b1303 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:23:59.654137 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f876b4-d66f-49cf-ba48-133a111b1303 None admin] PATH_INFO: `/v3/projects/85f4644d3bc94e89a0061a116652482e` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:23:59.655360 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55f876b4-d66f-49cf-ba48-133a111b1303 None admin] RBAC: Authorizing `identity:get_project(project_id=85f4644d3bc94e89a0061a116652482e)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:23:59.656842 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55f876b4-d66f-49cf-ba48-133a111b1303 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:23:59.659053 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 204/412] 10.222.0.33 () {56 vars in 1186 bytes} [Mon Jul 25 05:23:59 2022] GET /identity/v3/projects/85f4644d3bc94e89a0061a116652482e => generated 344 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:23:59.664517 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f876b4-d66f-49cf-ba48-133a111b1303 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:23:59.713157 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:23:59.733515 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54fbf3b6-ab0b-4567-9ea8-3ee01dc9d83f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:23:59.734401 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54fbf3b6-ab0b-4567-9ea8-3ee01dc9d83f None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:23:59.734686 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54fbf3b6-ab0b-4567-9ea8-3ee01dc9d83f None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:23:59.734995 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54fbf3b6-ab0b-4567-9ea8-3ee01dc9d83f None admin] PATH_INFO: `/v3/projects/ec1ed227a23f453cab64e15036af0c60` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:23:59.736198 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54fbf3b6-ab0b-4567-9ea8-3ee01dc9d83f None admin] RBAC: Authorizing `identity:get_project(project_id=ec1ed227a23f453cab64e15036af0c60)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:23:59.737668 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54fbf3b6-ab0b-4567-9ea8-3ee01dc9d83f None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:23:59.739673 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 209/413] 10.222.0.33 () {56 vars in 1186 bytes} [Mon Jul 25 05:23:59 2022] GET /identity/v3/projects/ec1ed227a23f453cab64e15036af0c60 => generated 302 bytes in 459 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 05:23:59.745208 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54fbf3b6-ab0b-4567-9ea8-3ee01dc9d83f None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:23:59.774077 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f876b4-d66f-49cf-ba48-133a111b1303 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:23:59.812921 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-724d03a3-277e-4c3a-b89a-09193af101bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:23:59.814321 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724d03a3-277e-4c3a-b89a-09193af101bc None admin] REQUEST_METHOD: `PUT` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:23:59.814615 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724d03a3-277e-4c3a-b89a-09193af101bc None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:23:59.814898 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724d03a3-277e-4c3a-b89a-09193af101bc None admin] PATH_INFO: `/v3/projects/85f4644d3bc94e89a0061a116652482e/groups/f6f19f9918714f0093cad025b7a07672/roles/39e5c666285a42fe8fc2c3afca6260d1` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:23:59.817013 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-724d03a3-277e-4c3a-b89a-09193af101bc None admin] RBAC: Authorizing `identity:create_grant(project_id=85f4644d3bc94e89a0061a116652482e, group_id=f6f19f9918714f0093cad025b7a07672, role_id=39e5c666285a42fe8fc2c3afca6260d1)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:23:59.818537 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-724d03a3-277e-4c3a-b89a-09193af101bc None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:23:59.834364 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 205/414] 10.222.0.33 () {58 vars in 1442 bytes} [Mon Jul 25 05:23:59 2022] PUT /identity/v3/projects/85f4644d3bc94e89a0061a116652482e/groups/f6f19f9918714f0093cad025b7a07672/roles/39e5c666285a42fe8fc2c3afca6260d1 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 25 05:23:59.962919 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:23:59.975120 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54fbf3b6-ab0b-4567-9ea8-3ee01dc9d83f None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:24:00.014180 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-698cf20e-2412-42cc-bdf3-066595805a69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:24:00.015084 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698cf20e-2412-42cc-bdf3-066595805a69 None admin] REQUEST_METHOD: `PUT` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:24:00.015374 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698cf20e-2412-42cc-bdf3-066595805a69 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:24:00.015654 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698cf20e-2412-42cc-bdf3-066595805a69 None admin] PATH_INFO: `/v3/projects/ec1ed227a23f453cab64e15036af0c60/groups/2673ac268143479dafd556fc6b3bc764/roles/02aba1f7cbdf4d6595e18af7fb5727e0` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:24:00.018103 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-698cf20e-2412-42cc-bdf3-066595805a69 None admin] RBAC: Authorizing `identity:create_grant(project_id=ec1ed227a23f453cab64e15036af0c60, group_id=2673ac268143479dafd556fc6b3bc764, role_id=02aba1f7cbdf4d6595e18af7fb5727e0)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:24:00.019593 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-698cf20e-2412-42cc-bdf3-066595805a69 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:24:00.036018 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 210/415] 10.222.0.33 () {58 vars in 1442 bytes} [Mon Jul 25 05:23:59 2022] PUT /identity/v3/projects/ec1ed227a23f453cab64e15036af0c60/groups/2673ac268143479dafd556fc6b3bc764/roles/02aba1f7cbdf4d6595e18af7fb5727e0 => generated 0 bytes in 291 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 05:24:00.072626 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d8370e-1539-45da-931f-623a3e9c1cb3 None None] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:24:00.073150 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d8370e-1539-45da-931f-623a3e9c1cb3 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:24:00.073150 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d8370e-1539-45da-931f-623a3e9c1cb3 None None] PATH_INFO: `/` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:24:00.074544 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 206/416] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jul 25 05:24:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:24:00.080642 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d10280-13ce-4d86-82dd-787fa0027c62 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:24:00.080980 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d10280-13ce-4d86-82dd-787fa0027c62 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:24:00.081251 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d10280-13ce-4d86-82dd-787fa0027c62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:24:00.101614 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-b3d10280-13ce-4d86-82dd-787fa0027c62 None None] MFA Rules not processed for user `1eae540242d94c10b0ad33960a855f08`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:24:00.102916 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-b3d10280-13ce-4d86-82dd-787fa0027c62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:24:00.192208 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 211/417] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jul 25 05:24:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 05:24:00.216570 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d8370e-1539-45da-931f-623a3e9c1cb3 None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:24:00.320468 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:24:00.329461 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d8370e-1539-45da-931f-623a3e9c1cb3 None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:24:00.365826 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e86cbcd-dcd2-4804-ada9-70bac448b64a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:24:00.366640 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e86cbcd-dcd2-4804-ada9-70bac448b64a None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:24:00.366925 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e86cbcd-dcd2-4804-ada9-70bac448b64a None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:24:00.367203 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e86cbcd-dcd2-4804-ada9-70bac448b64a None admin] PATH_INFO: `/v3/roles/member` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:24:00.378348 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e86cbcd-dcd2-4804-ada9-70bac448b64a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 05:24:00.386338 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application [None req-3e86cbcd-dcd2-4804-ada9-70bac448b64a None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 25 05:24:00.386338 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:24:00.386338 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:24:00.386338 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:24:00.386338 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:24:00.386338 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:24:00.386338 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:24:00.386338 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:24:00.386338 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:24:00.386338 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:24:00.386338 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:24:00.386338 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:24:00.386338 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 05:24:00.386338 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 05:24:00.386338 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 05:24:00.386338 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 05:24:00.387321 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 05:24:00.387321 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 05:24:00.387321 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 05:24:00.387321 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 05:24:00.387321 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:24:00.387321 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:24:00.387321 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:24:00.387321 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:24:00.387321 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:24:00.387321 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:24:00.387321 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:24:00.387321 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application with Lock( Jul 25 05:24:00.387321 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:24:00.387321 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:24:00.387321 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:24:00.387321 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:24:00.389285 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:24:00.389285 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:24:00.389285 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:24:00.389285 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:24:00.389285 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 05:24:00.389285 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 05:24:00.389285 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 05:24:00.389285 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 05:24:00.389285 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 05:24:00.389285 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 05:24:00.389285 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 25 05:24:00.389285 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Jul 25 05:24:00.389285 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 207/418] 10.222.0.33 () {56 vars in 1099 bytes} [Mon Jul 25 05:24:00 2022] GET /identity/v3/roles/member => generated 84 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:24:00.394403 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d10280-13ce-4d86-82dd-787fa0027c62 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:24:00.486642 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:24:00.502196 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d10280-13ce-4d86-82dd-787fa0027c62 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:24:00.538700 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b369043b-70d6-43cc-9697-2b5ac237e4b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:24:00.539383 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b369043b-70d6-43cc-9697-2b5ac237e4b2 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:24:00.539674 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b369043b-70d6-43cc-9697-2b5ac237e4b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:24:00.540057 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b369043b-70d6-43cc-9697-2b5ac237e4b2 None admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:24:00.540951 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b369043b-70d6-43cc-9697-2b5ac237e4b2 None admin] RBAC: Adding query filter params (name=member) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:24:00.541335 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b369043b-70d6-43cc-9697-2b5ac237e4b2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:24:00.542728 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b369043b-70d6-43cc-9697-2b5ac237e4b2 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:24:00.552191 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 212/419] 10.222.0.33 () {56 vars in 1101 bytes} [Mon Jul 25 05:24:00 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:24:00.557914 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e86cbcd-dcd2-4804-ada9-70bac448b64a None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:24:00.665040 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e86cbcd-dcd2-4804-ada9-70bac448b64a None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:24:00.700796 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f23377d3-2c45-4821-be2b-898cb3f0db1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:24:00.701542 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23377d3-2c45-4821-be2b-898cb3f0db1c None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:24:00.702160 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23377d3-2c45-4821-be2b-898cb3f0db1c None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:24:00.702445 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23377d3-2c45-4821-be2b-898cb3f0db1c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:24:00.727918 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:24:00.774414 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f23377d3-2c45-4821-be2b-898cb3f0db1c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:24:00.775892 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f23377d3-2c45-4821-be2b-898cb3f0db1c None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:24:00.850497 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca662f0-2a70-450c-be77-78a66855d089 None None] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:24:00.850807 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca662f0-2a70-450c-be77-78a66855d089 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:24:00.851069 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca662f0-2a70-450c-be77-78a66855d089 None None] PATH_INFO: `/` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:24:00.852715 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 213/420] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jul 25 05:24:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:24:00.862187 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec6eb1a-daa3-4cbd-939b-4f57ad2f2288 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:24:00.862606 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec6eb1a-daa3-4cbd-939b-4f57ad2f2288 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:24:00.863125 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec6eb1a-daa3-4cbd-939b-4f57ad2f2288 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:24:00.895903 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-eec6eb1a-daa3-4cbd-939b-4f57ad2f2288 None None] MFA Rules not processed for user `1eae540242d94c10b0ad33960a855f08`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:24:00.897146 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-eec6eb1a-daa3-4cbd-939b-4f57ad2f2288 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:24:00.903633 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 208/421] 10.222.0.33 () {58 vars in 1282 bytes} [Mon Jul 25 05:24:00 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 346 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 25 05:24:00.910178 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f23377d3-2c45-4821-be2b-898cb3f0db1c None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:24:01.014101 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:24:01.025969 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 214/422] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jul 25 05:24:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 25 05:24:01.064868 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec6eb1a-daa3-4cbd-939b-4f57ad2f2288 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:24:01.074701 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f23377d3-2c45-4821-be2b-898cb3f0db1c None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:24:01.146247 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc085c5-5748-43ff-8a46-91d3e451565b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:24:01.147232 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc085c5-5748-43ff-8a46-91d3e451565b None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:24:01.147651 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc085c5-5748-43ff-8a46-91d3e451565b None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:24:01.147917 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc085c5-5748-43ff-8a46-91d3e451565b None admin] PATH_INFO: `/v3/projects/4b840bd0540b4512b180fc68fd41cedc` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:24:01.151386 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bc085c5-5748-43ff-8a46-91d3e451565b None admin] RBAC: Authorizing `identity:get_project(project_id=4b840bd0540b4512b180fc68fd41cedc)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:24:01.152883 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bc085c5-5748-43ff-8a46-91d3e451565b None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:24:01.155400 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 209/423] 10.222.0.33 () {56 vars in 1186 bytes} [Mon Jul 25 05:24:00 2022] GET /identity/v3/projects/4b840bd0540b4512b180fc68fd41cedc => generated 298 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 05:24:01.160334 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc085c5-5748-43ff-8a46-91d3e451565b None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:24:01.184399 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:24:01.214617 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec6eb1a-daa3-4cbd-939b-4f57ad2f2288 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:24:01.258579 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87efbf50-cb76-4cba-8c97-1611f578d97f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:24:01.259885 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87efbf50-cb76-4cba-8c97-1611f578d97f None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:24:01.260384 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87efbf50-cb76-4cba-8c97-1611f578d97f None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:24:01.260868 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87efbf50-cb76-4cba-8c97-1611f578d97f None admin] PATH_INFO: `/v3/roles/member` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:24:01.275460 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87efbf50-cb76-4cba-8c97-1611f578d97f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 05:24:01.275768 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc085c5-5748-43ff-8a46-91d3e451565b None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:24:01.285196 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application [None req-87efbf50-cb76-4cba-8c97-1611f578d97f None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 25 05:24:01.285196 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:24:01.285196 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:24:01.285196 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:24:01.285196 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:24:01.285196 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:24:01.285196 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:24:01.285196 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:24:01.285196 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:24:01.285196 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:24:01.285196 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:24:01.285196 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:24:01.285196 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 05:24:01.285196 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 05:24:01.285196 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 05:24:01.285196 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 05:24:01.286110 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 05:24:01.286110 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 05:24:01.286110 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 05:24:01.286110 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 05:24:01.286110 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:24:01.286110 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:24:01.286110 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:24:01.286110 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:24:01.286110 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:24:01.286110 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:24:01.286110 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:24:01.286110 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application with Lock( Jul 25 05:24:01.286110 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:24:01.286110 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:24:01.286110 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:24:01.286110 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:24:01.287568 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:24:01.287568 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:24:01.287568 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:24:01.287568 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:24:01.287568 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 05:24:01.287568 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 05:24:01.287568 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 05:24:01.287568 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 05:24:01.287568 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 05:24:01.287568 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 05:24:01.287568 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 25 05:24:01.287568 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Jul 25 05:24:01.287568 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 215/424] 10.222.0.33 () {56 vars in 1099 bytes} [Mon Jul 25 05:24:01 2022] GET /identity/v3/roles/member => generated 84 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 25 05:24:01.288240 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b32d1a2-4d3f-4442-8aa7-405589dc786a None None] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:24:01.288307 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b32d1a2-4d3f-4442-8aa7-405589dc786a None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:24:01.288567 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b32d1a2-4d3f-4442-8aa7-405589dc786a None None] PATH_INFO: `/` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:24:01.294770 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 216/425] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jul 25 05:24:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:24:01.294770 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b32d1a2-4d3f-4442-8aa7-405589dc786a None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:24:01.308577 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:24:01.322669 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-826b6e44-94c6-42c2-96be-486f56b2ed1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:24:01.323982 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826b6e44-94c6-42c2-96be-486f56b2ed1a None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:24:01.324363 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826b6e44-94c6-42c2-96be-486f56b2ed1a None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:24:01.324743 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826b6e44-94c6-42c2-96be-486f56b2ed1a None admin] PATH_INFO: `/v3/groups/2673ac268143479dafd556fc6b3bc764` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:24:01.326398 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-826b6e44-94c6-42c2-96be-486f56b2ed1a None admin] RBAC: Authorizing `identity:get_group(group_id=2673ac268143479dafd556fc6b3bc764)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:24:01.328030 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-826b6e44-94c6-42c2-96be-486f56b2ed1a None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:24:01.334950 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 210/426] 10.222.0.33 () {56 vars in 1180 bytes} [Mon Jul 25 05:24:01 2022] GET /identity/v3/groups/2673ac268143479dafd556fc6b3bc764 => generated 226 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 05:24:01.336528 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990629c9-9c17-4cf4-be8c-7b89aa4d0178 None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:24:01.337162 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990629c9-9c17-4cf4-be8c-7b89aa4d0178 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:24:01.337555 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990629c9-9c17-4cf4-be8c-7b89aa4d0178 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:24:01.357851 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-990629c9-9c17-4cf4-be8c-7b89aa4d0178 None None] MFA Rules not processed for user `1eae540242d94c10b0ad33960a855f08`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:24:01.359211 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-990629c9-9c17-4cf4-be8c-7b89aa4d0178 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:24:01.436724 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 211/427] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jul 25 05:24:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 05:24:01.437500 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b32d1a2-4d3f-4442-8aa7-405589dc786a None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:24:01.437914 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-990629c9-9c17-4cf4-be8c-7b89aa4d0178 None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:24:01.483399 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee2db372-5f9b-47fc-a5ae-dd7fc9b5d5b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:24:01.484245 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2db372-5f9b-47fc-a5ae-dd7fc9b5d5b1 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:24:01.484638 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2db372-5f9b-47fc-a5ae-dd7fc9b5d5b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:24:01.485195 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2db372-5f9b-47fc-a5ae-dd7fc9b5d5b1 None admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:24:01.485947 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee2db372-5f9b-47fc-a5ae-dd7fc9b5d5b1 None admin] RBAC: Adding query filter params (name=member) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:24:01.486343 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee2db372-5f9b-47fc-a5ae-dd7fc9b5d5b1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:24:01.487834 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee2db372-5f9b-47fc-a5ae-dd7fc9b5d5b1 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:24:01.493235 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:24:01.497559 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 217/428] 10.222.0.33 () {56 vars in 1101 bytes} [Mon Jul 25 05:24:01 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:24:01.499162 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee2db372-5f9b-47fc-a5ae-dd7fc9b5d5b1 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:24:01.567506 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-990629c9-9c17-4cf4-be8c-7b89aa4d0178 None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:24:01.589589 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:24:01.607780 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-432b0bda-27ec-4e07-b7e4-1e83b4b0d7f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:24:01.608442 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432b0bda-27ec-4e07-b7e4-1e83b4b0d7f7 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:24:01.608730 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432b0bda-27ec-4e07-b7e4-1e83b4b0d7f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:24:01.609040 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432b0bda-27ec-4e07-b7e4-1e83b4b0d7f7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:24:01.610001 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-432b0bda-27ec-4e07-b7e4-1e83b4b0d7f7 None admin] RBAC: Adding query filter params (group.id=2673ac268143479dafd556fc6b3bc764, role.id=5074e74e35f54d4b9bb415c34999a82a, scope.project.id=4b840bd0540b4512b180fc68fd41cedc) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:24:01.610318 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-432b0bda-27ec-4e07-b7e4-1e83b4b0d7f7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:24:01.611887 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-432b0bda-27ec-4e07-b7e4-1e83b4b0d7f7 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:24:01.614289 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee2db372-5f9b-47fc-a5ae-dd7fc9b5d5b1 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:24:01.624815 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 212/429] 10.222.0.33 () {56 vars in 1376 bytes} [Mon Jul 25 05:24:01 2022] GET /identity/v3/role_assignments?group.id=2673ac268143479dafd556fc6b3bc764&scope.project.id=4b840bd0540b4512b180fc68fd41cedc&role.id=5074e74e35f54d4b9bb415c34999a82a => generated 611 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:24:01.624815 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-432b0bda-27ec-4e07-b7e4-1e83b4b0d7f7 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:24:01.691088 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9919583f-98b6-4000-be58-fa27a4c8a2b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:24:01.692287 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9919583f-98b6-4000-be58-fa27a4c8a2b3 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:24:01.692718 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9919583f-98b6-4000-be58-fa27a4c8a2b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:24:01.693158 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9919583f-98b6-4000-be58-fa27a4c8a2b3 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:24:01.716381 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9919583f-98b6-4000-be58-fa27a4c8a2b3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 05:24:01.733493 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application [None req-9919583f-98b6-4000-be58-fa27a4c8a2b3 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 25 05:24:01.733493 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:24:01.733493 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:24:01.733493 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:24:01.733493 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:24:01.733493 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:24:01.733493 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:24:01.733493 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:24:01.733493 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:24:01.733493 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:24:01.733493 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:24:01.733493 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:24:01.733493 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 05:24:01.733493 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 05:24:01.733493 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 05:24:01.733493 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 05:24:01.734646 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 05:24:01.734646 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 05:24:01.734646 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 05:24:01.734646 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 05:24:01.734646 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:24:01.734646 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:24:01.734646 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:24:01.734646 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:24:01.734646 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:24:01.734646 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:24:01.734646 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:24:01.734646 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application with Lock( Jul 25 05:24:01.734646 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:24:01.734646 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:24:01.734646 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:24:01.734646 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:24:01.735649 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:24:01.735649 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:24:01.735649 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:24:01.735649 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:24:01.735649 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 05:24:01.735649 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 05:24:01.735649 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 05:24:01.735649 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 05:24:01.735649 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 05:24:01.735649 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 05:24:01.735649 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 25 05:24:01.735649 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Jul 25 05:24:01.738473 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 218/430] 10.222.0.33 () {56 vars in 1114 bytes} [Mon Jul 25 05:24:01 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:24:01.744277 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-432b0bda-27ec-4e07-b7e4-1e83b4b0d7f7 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:24:01.744711 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9919583f-98b6-4000-be58-fa27a4c8a2b3 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:24:01.782227 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d3b839-566d-4920-97d0-0b1928e9c5f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:24:01.782989 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d3b839-566d-4920-97d0-0b1928e9c5f0 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:24:01.783348 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d3b839-566d-4920-97d0-0b1928e9c5f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:24:01.783694 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d3b839-566d-4920-97d0-0b1928e9c5f0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:24:01.844554 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:24:01.852558 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:24:01.865416 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68d3b839-566d-4920-97d0-0b1928e9c5f0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:24:01.867309 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68d3b839-566d-4920-97d0-0b1928e9c5f0 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:24:01.915062 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9919583f-98b6-4000-be58-fa27a4c8a2b3 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:24:01.953570 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f855d97-63be-48ea-ac98-15af05665fce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:24:01.954252 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f855d97-63be-48ea-ac98-15af05665fce None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:24:01.954543 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f855d97-63be-48ea-ac98-15af05665fce None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:24:01.954829 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f855d97-63be-48ea-ac98-15af05665fce None admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:24:01.956405 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f855d97-63be-48ea-ac98-15af05665fce None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:24:01.956714 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f855d97-63be-48ea-ac98-15af05665fce None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:24:01.958142 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f855d97-63be-48ea-ac98-15af05665fce None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:24:01.967491 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 219/431] 10.222.0.33 () {56 vars in 1111 bytes} [Mon Jul 25 05:24:01 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 05:24:01.973556 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f855d97-63be-48ea-ac98-15af05665fce None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:24:01.980014 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 213/432] 10.222.0.33 () {58 vars in 1282 bytes} [Mon Jul 25 05:24:01 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 361 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 25 05:24:01.986365 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d3b839-566d-4920-97d0-0b1928e9c5f0 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:24:02.079212 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:24:02.093921 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f855d97-63be-48ea-ac98-15af05665fce None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:24:02.099054 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d3b839-566d-4920-97d0-0b1928e9c5f0 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:24:02.119887 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:24:02.134800 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b9e653-f61b-4542-8623-6994aca7aed4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:24:02.135446 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b9e653-f61b-4542-8623-6994aca7aed4 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:24:02.135739 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b9e653-f61b-4542-8623-6994aca7aed4 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:24:02.136036 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b9e653-f61b-4542-8623-6994aca7aed4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:24:02.139002 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aefa8563-6022-42c9-8a74-dfb6995cbfdb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:24:02.140100 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aefa8563-6022-42c9-8a74-dfb6995cbfdb None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:24:02.140599 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aefa8563-6022-42c9-8a74-dfb6995cbfdb None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:24:02.141121 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aefa8563-6022-42c9-8a74-dfb6995cbfdb None admin] PATH_INFO: `/v3/projects/ec1ed227a23f453cab64e15036af0c60` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:24:02.142543 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aefa8563-6022-42c9-8a74-dfb6995cbfdb None admin] RBAC: Authorizing `identity:get_project(project_id=ec1ed227a23f453cab64e15036af0c60)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:24:02.144206 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aefa8563-6022-42c9-8a74-dfb6995cbfdb None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:24:02.146459 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 214/433] 10.222.0.33 () {56 vars in 1186 bytes} [Mon Jul 25 05:24:01 2022] GET /identity/v3/projects/ec1ed227a23f453cab64e15036af0c60 => generated 302 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 05:24:02.152646 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aefa8563-6022-42c9-8a74-dfb6995cbfdb None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:24:02.211583 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42b9e653-f61b-4542-8623-6994aca7aed4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:24:02.212819 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42b9e653-f61b-4542-8623-6994aca7aed4 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:24:02.264212 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aefa8563-6022-42c9-8a74-dfb6995cbfdb None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:24:02.301646 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16bfb9e8-9c04-478c-b4fd-88d0271b8180 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:24:02.303060 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16bfb9e8-9c04-478c-b4fd-88d0271b8180 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:24:02.303544 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16bfb9e8-9c04-478c-b4fd-88d0271b8180 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:24:02.304016 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16bfb9e8-9c04-478c-b4fd-88d0271b8180 None admin] PATH_INFO: `/v3/groups/2673ac268143479dafd556fc6b3bc764` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:24:02.305623 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16bfb9e8-9c04-478c-b4fd-88d0271b8180 None admin] RBAC: Authorizing `identity:get_group(group_id=2673ac268143479dafd556fc6b3bc764)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:24:02.307335 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16bfb9e8-9c04-478c-b4fd-88d0271b8180 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:24:02.310234 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 215/434] 10.222.0.33 () {56 vars in 1180 bytes} [Mon Jul 25 05:24:02 2022] GET /identity/v3/groups/2673ac268143479dafd556fc6b3bc764 => generated 226 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:24:02.319149 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16bfb9e8-9c04-478c-b4fd-88d0271b8180 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:24:02.334264 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 220/435] 10.222.0.33 () {58 vars in 1282 bytes} [Mon Jul 25 05:24:01 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 361 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 25 05:24:02.337350 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d91047-e949-4552-84aa-c519a4b41aa9 None None] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:24:02.337635 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d91047-e949-4552-84aa-c519a4b41aa9 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:24:02.337910 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d91047-e949-4552-84aa-c519a4b41aa9 None None] PATH_INFO: `/` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:24:02.342295 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 221/436] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jul 25 05:24:02 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:24:02.342295 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d91047-e949-4552-84aa-c519a4b41aa9 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:24:02.352565 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:24:02.367317 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:24:02.456618 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16bfb9e8-9c04-478c-b4fd-88d0271b8180 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:24:02.478875 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d91047-e949-4552-84aa-c519a4b41aa9 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:24:02.496956 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-896a6e29-2c97-4f32-9971-8fd12e3c2613 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:24:02.497655 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896a6e29-2c97-4f32-9971-8fd12e3c2613 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:24:02.497939 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896a6e29-2c97-4f32-9971-8fd12e3c2613 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:24:02.498219 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896a6e29-2c97-4f32-9971-8fd12e3c2613 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:24:02.499056 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-896a6e29-2c97-4f32-9971-8fd12e3c2613 None admin] RBAC: Adding query filter params (group.id=2673ac268143479dafd556fc6b3bc764, role.id=5074e74e35f54d4b9bb415c34999a82a, scope.project.id=ec1ed227a23f453cab64e15036af0c60) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:24:02.499487 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-896a6e29-2c97-4f32-9971-8fd12e3c2613 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:24:02.501714 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-896a6e29-2c97-4f32-9971-8fd12e3c2613 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:24:02.511212 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 216/437] 10.222.0.33 () {56 vars in 1376 bytes} [Mon Jul 25 05:24:02 2022] GET /identity/v3/role_assignments?group.id=2673ac268143479dafd556fc6b3bc764&scope.project.id=ec1ed227a23f453cab64e15036af0c60&role.id=5074e74e35f54d4b9bb415c34999a82a => generated 611 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 05:24:02.512507 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66bf6ad2-3580-4a60-a39d-d58d4eb34fc0 None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:24:02.513395 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66bf6ad2-3580-4a60-a39d-d58d4eb34fc0 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:24:02.513683 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66bf6ad2-3580-4a60-a39d-d58d4eb34fc0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:24:02.523981 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f9f9cc7-cdaa-4951-9b0f-a929cca77554 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:24:02.524922 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9f9cc7-cdaa-4951-9b0f-a929cca77554 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:24:02.525210 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9f9cc7-cdaa-4951-9b0f-a929cca77554 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:24:02.525490 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9f9cc7-cdaa-4951-9b0f-a929cca77554 None admin] PATH_INFO: `/v3/projects/4b840bd0540b4512b180fc68fd41cedc` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:24:02.526780 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f9f9cc7-cdaa-4951-9b0f-a929cca77554 None admin] RBAC: Authorizing `identity:get_project(project_id=4b840bd0540b4512b180fc68fd41cedc)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:24:02.528399 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f9f9cc7-cdaa-4951-9b0f-a929cca77554 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:24:02.530531 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 222/438] 10.222.0.33 () {56 vars in 1186 bytes} [Mon Jul 25 05:24:02 2022] GET /identity/v3/projects/4b840bd0540b4512b180fc68fd41cedc => generated 298 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:24:02.533646 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-66bf6ad2-3580-4a60-a39d-d58d4eb34fc0 None None] MFA Rules not processed for user `1eae540242d94c10b0ad33960a855f08`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:24:02.534945 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-66bf6ad2-3580-4a60-a39d-d58d4eb34fc0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:24:02.537847 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f9f9cc7-cdaa-4951-9b0f-a929cca77554 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:24:02.612868 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 217/439] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jul 25 05:24:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 05:24:02.636437 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66bf6ad2-3580-4a60-a39d-d58d4eb34fc0 None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:24:02.647578 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:24:02.661896 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:24:02.662126 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f9f9cc7-cdaa-4951-9b0f-a929cca77554 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:24:02.703630 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7794d053-a1dc-44e5-9bbd-570811486562 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:24:02.704409 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7794d053-a1dc-44e5-9bbd-570811486562 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:24:02.704698 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7794d053-a1dc-44e5-9bbd-570811486562 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:24:02.705097 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7794d053-a1dc-44e5-9bbd-570811486562 None admin] PATH_INFO: `/v3/groups/2673ac268143479dafd556fc6b3bc764` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:24:02.706422 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7794d053-a1dc-44e5-9bbd-570811486562 None admin] RBAC: Authorizing `identity:get_group(group_id=2673ac268143479dafd556fc6b3bc764)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:24:02.708085 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7794d053-a1dc-44e5-9bbd-570811486562 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:24:02.710320 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 223/440] 10.222.0.33 () {56 vars in 1180 bytes} [Mon Jul 25 05:24:02 2022] GET /identity/v3/groups/2673ac268143479dafd556fc6b3bc764 => generated 226 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:24:02.717467 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7794d053-a1dc-44e5-9bbd-570811486562 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:24:02.787043 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66bf6ad2-3580-4a60-a39d-d58d4eb34fc0 None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:24:02.822164 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a531f306-35e6-49d2-a405-539d8d13f0db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:24:02.822964 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a531f306-35e6-49d2-a405-539d8d13f0db None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:24:02.823250 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a531f306-35e6-49d2-a405-539d8d13f0db None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:24:02.823530 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a531f306-35e6-49d2-a405-539d8d13f0db None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:24:02.834754 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a531f306-35e6-49d2-a405-539d8d13f0db None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 05:24:02.839405 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7794d053-a1dc-44e5-9bbd-570811486562 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:24:02.842574 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application [None req-a531f306-35e6-49d2-a405-539d8d13f0db None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 05:24:02.842574 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:24:02.842574 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:24:02.842574 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:24:02.842574 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:24:02.842574 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:24:02.842574 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:24:02.842574 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:24:02.842574 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:24:02.842574 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:24:02.842574 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:24:02.842574 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:24:02.842574 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 05:24:02.842574 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 05:24:02.842574 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 05:24:02.842574 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 05:24:02.843583 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 05:24:02.843583 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 05:24:02.843583 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 05:24:02.843583 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 05:24:02.843583 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:24:02.843583 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:24:02.843583 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:24:02.843583 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:24:02.843583 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:24:02.843583 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:24:02.843583 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:24:02.843583 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application with Lock( Jul 25 05:24:02.843583 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:24:02.843583 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:24:02.843583 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:24:02.843583 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:24:02.844400 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:24:02.844400 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:24:02.844400 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:24:02.844400 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:24:02.844400 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 05:24:02.844400 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 05:24:02.844400 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 05:24:02.844400 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 05:24:02.844400 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 05:24:02.844400 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 05:24:02.844400 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 05:24:02.844400 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Jul 25 05:24:02.844400 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 218/441] 10.222.0.33 () {56 vars in 1096 bytes} [Mon Jul 25 05:24:02 2022] GET /identity/v3/roles/admin => generated 83 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 25 05:24:02.845490 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fce28d4-255c-47ce-9f86-54f1bac5a3c8 None None] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:24:02.845691 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fce28d4-255c-47ce-9f86-54f1bac5a3c8 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:24:02.845971 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fce28d4-255c-47ce-9f86-54f1bac5a3c8 None None] PATH_INFO: `/` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:24:02.847230 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 219/442] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jul 25 05:24:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:24:02.849788 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fce28d4-255c-47ce-9f86-54f1bac5a3c8 None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:24:02.875679 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-050da711-731d-42b4-bba5-c64bbee98559 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:24:02.876264 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050da711-731d-42b4-bba5-c64bbee98559 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:24:02.876547 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050da711-731d-42b4-bba5-c64bbee98559 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:24:02.876853 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050da711-731d-42b4-bba5-c64bbee98559 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:24:02.877687 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-050da711-731d-42b4-bba5-c64bbee98559 None admin] RBAC: Adding query filter params (group.id=2673ac268143479dafd556fc6b3bc764, role.id=02aba1f7cbdf4d6595e18af7fb5727e0, scope.project.id=4b840bd0540b4512b180fc68fd41cedc) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:24:02.877998 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-050da711-731d-42b4-bba5-c64bbee98559 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:24:02.879420 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-050da711-731d-42b4-bba5-c64bbee98559 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:24:02.900809 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 224/443] 10.222.0.33 () {56 vars in 1376 bytes} [Mon Jul 25 05:24:02 2022] GET /identity/v3/role_assignments?group.id=2673ac268143479dafd556fc6b3bc764&scope.project.id=4b840bd0540b4512b180fc68fd41cedc&role.id=02aba1f7cbdf4d6595e18af7fb5727e0 => generated 611 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 05:24:02.900809 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22bff6a-e9ee-4a36-81d4-65099bf04152 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:24:02.900809 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22bff6a-e9ee-4a36-81d4-65099bf04152 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:24:02.900809 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22bff6a-e9ee-4a36-81d4-65099bf04152 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:24:02.914938 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-f22bff6a-e9ee-4a36-81d4-65099bf04152 None None] MFA Rules not processed for user `1eae540242d94c10b0ad33960a855f08`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:24:02.916164 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-f22bff6a-e9ee-4a36-81d4-65099bf04152 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:24:02.931848 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:24:02.936768 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:24:02.962836 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fce28d4-255c-47ce-9f86-54f1bac5a3c8 None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:24:02.998893 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 225/444] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jul 25 05:24:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 05:24:03.000662 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df7d1ba6-d9f6-4af8-9edd-32c019ed4eff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:24:03.001381 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7d1ba6-d9f6-4af8-9edd-32c019ed4eff None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:24:03.001720 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7d1ba6-d9f6-4af8-9edd-32c019ed4eff None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:24:03.002000 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7d1ba6-d9f6-4af8-9edd-32c019ed4eff None admin] PATH_INFO: `/v3/roles` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:24:03.002806 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df7d1ba6-d9f6-4af8-9edd-32c019ed4eff None admin] RBAC: Adding query filter params (name=admin) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:24:03.003110 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df7d1ba6-d9f6-4af8-9edd-32c019ed4eff None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:24:03.004534 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df7d1ba6-d9f6-4af8-9edd-32c019ed4eff None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:24:03.014676 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 220/445] 10.222.0.33 () {56 vars in 1099 bytes} [Mon Jul 25 05:24:02 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 05:24:03.020180 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f22bff6a-e9ee-4a36-81d4-65099bf04152 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:24:03.026647 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df7d1ba6-d9f6-4af8-9edd-32c019ed4eff None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:24:03.128846 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f22bff6a-e9ee-4a36-81d4-65099bf04152 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:24:03.150122 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df7d1ba6-d9f6-4af8-9edd-32c019ed4eff None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:24:03.169373 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a957b9-72f5-4252-a5cf-c7e26af93bb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:24:03.170338 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a957b9-72f5-4252-a5cf-c7e26af93bb2 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:24:03.170803 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a957b9-72f5-4252-a5cf-c7e26af93bb2 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:24:03.171219 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a957b9-72f5-4252-a5cf-c7e26af93bb2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:24:03.190692 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e014cd51-bd9d-4952-a905-852899cf53b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:24:03.191517 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e014cd51-bd9d-4952-a905-852899cf53b7 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:24:03.191807 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e014cd51-bd9d-4952-a905-852899cf53b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:24:03.192122 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e014cd51-bd9d-4952-a905-852899cf53b7 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:24:03.195374 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:24:03.197364 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:24:03.206073 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e014cd51-bd9d-4952-a905-852899cf53b7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 05:24:03.216700 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application [None req-e014cd51-bd9d-4952-a905-852899cf53b7 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 25 05:24:03.216700 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:24:03.216700 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:24:03.216700 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:24:03.216700 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:24:03.216700 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:24:03.216700 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:24:03.216700 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:24:03.216700 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:24:03.216700 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:24:03.216700 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:24:03.216700 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:24:03.216700 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 05:24:03.216700 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 05:24:03.216700 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 05:24:03.216700 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 05:24:03.217703 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 05:24:03.217703 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 05:24:03.217703 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 05:24:03.217703 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 05:24:03.217703 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:24:03.217703 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:24:03.217703 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:24:03.217703 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:24:03.217703 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:24:03.217703 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:24:03.217703 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:24:03.217703 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application with Lock( Jul 25 05:24:03.217703 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:24:03.217703 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:24:03.217703 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:24:03.217703 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:24:03.218507 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:24:03.218507 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:24:03.218507 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:24:03.218507 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:24:03.218507 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 05:24:03.218507 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 05:24:03.218507 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 05:24:03.218507 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 05:24:03.218507 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 05:24:03.218507 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 05:24:03.218507 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 25 05:24:03.218507 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Jul 25 05:24:03.218507 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 221/446] 10.222.0.33 () {56 vars in 1114 bytes} [Mon Jul 25 05:24:03 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 25 05:24:03.222429 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e014cd51-bd9d-4952-a905-852899cf53b7 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:24:03.257007 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18a957b9-72f5-4252-a5cf-c7e26af93bb2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:24:03.258495 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18a957b9-72f5-4252-a5cf-c7e26af93bb2 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:24:03.343275 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e014cd51-bd9d-4952-a905-852899cf53b7 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:24:03.381643 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 226/447] 10.222.0.33 () {58 vars in 1282 bytes} [Mon Jul 25 05:24:03 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 362 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 25 05:24:03.388373 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a957b9-72f5-4252-a5cf-c7e26af93bb2 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:24:03.389178 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc8770e1-6389-4021-bdf6-8fc66c217dd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:24:03.389756 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8770e1-6389-4021-bdf6-8fc66c217dd9 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:24:03.390039 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8770e1-6389-4021-bdf6-8fc66c217dd9 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:24:03.390313 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8770e1-6389-4021-bdf6-8fc66c217dd9 None admin] PATH_INFO: `/v3/roles` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:24:03.391070 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc8770e1-6389-4021-bdf6-8fc66c217dd9 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:24:03.391374 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc8770e1-6389-4021-bdf6-8fc66c217dd9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:24:03.392878 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc8770e1-6389-4021-bdf6-8fc66c217dd9 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:24:03.403783 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 222/448] 10.222.0.33 () {56 vars in 1111 bytes} [Mon Jul 25 05:24:03 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 05:24:03.409989 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc8770e1-6389-4021-bdf6-8fc66c217dd9 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:24:03.441079 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:24:03.472422 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:24:03.488377 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a957b9-72f5-4252-a5cf-c7e26af93bb2 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:24:03.514144 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc8770e1-6389-4021-bdf6-8fc66c217dd9 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:24:03.521095 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43817dc3-ffc1-470d-a3a6-150dc786f88d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:24:03.521949 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43817dc3-ffc1-470d-a3a6-150dc786f88d None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:24:03.522230 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43817dc3-ffc1-470d-a3a6-150dc786f88d None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:24:03.522504 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43817dc3-ffc1-470d-a3a6-150dc786f88d None admin] PATH_INFO: `/v3/projects/85f4644d3bc94e89a0061a116652482e` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:24:03.523676 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43817dc3-ffc1-470d-a3a6-150dc786f88d None admin] RBAC: Authorizing `identity:get_project(project_id=85f4644d3bc94e89a0061a116652482e)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:24:03.525143 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43817dc3-ffc1-470d-a3a6-150dc786f88d None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:24:03.526836 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 227/449] 10.222.0.33 () {56 vars in 1186 bytes} [Mon Jul 25 05:24:03 2022] GET /identity/v3/projects/85f4644d3bc94e89a0061a116652482e => generated 344 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 05:24:03.532918 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43817dc3-ffc1-470d-a3a6-150dc786f88d None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:24:03.549649 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dae1069-8ebd-4a65-a2b9-8e96b88aecb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:24:03.550303 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dae1069-8ebd-4a65-a2b9-8e96b88aecb3 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:24:03.550593 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dae1069-8ebd-4a65-a2b9-8e96b88aecb3 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:24:03.550871 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dae1069-8ebd-4a65-a2b9-8e96b88aecb3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:24:03.620624 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dae1069-8ebd-4a65-a2b9-8e96b88aecb3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:24:03.622115 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dae1069-8ebd-4a65-a2b9-8e96b88aecb3 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:24:03.632415 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43817dc3-ffc1-470d-a3a6-150dc786f88d None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:24:03.650983 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:24:03.665090 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-592ec0f7-e962-4900-9ac5-ae33b0291184 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:24:03.665934 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592ec0f7-e962-4900-9ac5-ae33b0291184 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:24:03.666216 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592ec0f7-e962-4900-9ac5-ae33b0291184 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:24:03.666492 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592ec0f7-e962-4900-9ac5-ae33b0291184 None admin] PATH_INFO: `/v3/groups/f6f19f9918714f0093cad025b7a07672` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:24:03.667653 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-592ec0f7-e962-4900-9ac5-ae33b0291184 None admin] RBAC: Authorizing `identity:get_group(group_id=f6f19f9918714f0093cad025b7a07672)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:24:03.669874 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-592ec0f7-e962-4900-9ac5-ae33b0291184 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:24:03.671497 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 228/450] 10.222.0.33 () {56 vars in 1180 bytes} [Mon Jul 25 05:24:03 2022] GET /identity/v3/groups/f6f19f9918714f0093cad025b7a07672 => generated 229 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 05:24:03.677210 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-592ec0f7-e962-4900-9ac5-ae33b0291184 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:24:03.679574 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:24:03.722761 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 223/451] 10.222.0.33 () {58 vars in 1282 bytes} [Mon Jul 25 05:24:03 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 25 05:24:03.728233 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dae1069-8ebd-4a65-a2b9-8e96b88aecb3 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:24:03.779653 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-592ec0f7-e962-4900-9ac5-ae33b0291184 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:24:03.816281 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb47f00d-07fb-4b10-8b64-0677a8c6e82f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:24:03.816978 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb47f00d-07fb-4b10-8b64-0677a8c6e82f None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:24:03.817271 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb47f00d-07fb-4b10-8b64-0677a8c6e82f None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:24:03.817556 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb47f00d-07fb-4b10-8b64-0677a8c6e82f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:24:03.818398 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb47f00d-07fb-4b10-8b64-0677a8c6e82f None admin] RBAC: Adding query filter params (group.id=f6f19f9918714f0093cad025b7a07672, role.id=39e5c666285a42fe8fc2c3afca6260d1, scope.project.id=85f4644d3bc94e89a0061a116652482e) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:24:03.818715 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb47f00d-07fb-4b10-8b64-0677a8c6e82f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:24:03.820157 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb47f00d-07fb-4b10-8b64-0677a8c6e82f None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:24:03.826631 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 229/452] 10.222.0.33 () {56 vars in 1376 bytes} [Mon Jul 25 05:24:03 2022] GET /identity/v3/role_assignments?group.id=f6f19f9918714f0093cad025b7a07672&scope.project.id=85f4644d3bc94e89a0061a116652482e&role.id=39e5c666285a42fe8fc2c3afca6260d1 => generated 611 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 05:24:03.827414 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dae1069-8ebd-4a65-a2b9-8e96b88aecb3 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:24:03.865270 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-162814f0-1277-4a22-aa3d-59b898197cd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:24:03.866144 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162814f0-1277-4a22-aa3d-59b898197cd4 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:24:03.866428 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162814f0-1277-4a22-aa3d-59b898197cd4 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:24:03.866704 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162814f0-1277-4a22-aa3d-59b898197cd4 None admin] PATH_INFO: `/v3/projects/ec1ed227a23f453cab64e15036af0c60` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:24:03.867931 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-162814f0-1277-4a22-aa3d-59b898197cd4 None admin] RBAC: Authorizing `identity:get_project(project_id=ec1ed227a23f453cab64e15036af0c60)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:24:03.869436 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-162814f0-1277-4a22-aa3d-59b898197cd4 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:24:03.871197 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 224/453] 10.222.0.33 () {56 vars in 1186 bytes} [Mon Jul 25 05:24:03 2022] GET /identity/v3/projects/ec1ed227a23f453cab64e15036af0c60 => generated 302 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 05:24:03.877786 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb47f00d-07fb-4b10-8b64-0677a8c6e82f None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:24:03.933820 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:24:03.983902 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb47f00d-07fb-4b10-8b64-0677a8c6e82f None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:24:04.021186 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de91764-2e66-4e46-942e-0f48c1d89f7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:24:04.021662 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de91764-2e66-4e46-942e-0f48c1d89f7f None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:24:04.021947 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de91764-2e66-4e46-942e-0f48c1d89f7f None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:24:04.022223 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de91764-2e66-4e46-942e-0f48c1d89f7f None admin] PATH_INFO: `/v3/groups/2673ac268143479dafd556fc6b3bc764` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:24:04.023403 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2de91764-2e66-4e46-942e-0f48c1d89f7f None admin] RBAC: Authorizing `identity:get_group(group_id=2673ac268143479dafd556fc6b3bc764)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:24:04.024921 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2de91764-2e66-4e46-942e-0f48c1d89f7f None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:24:04.026555 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 230/454] 10.222.0.33 () {56 vars in 1180 bytes} [Mon Jul 25 05:24:03 2022] GET /identity/v3/groups/2673ac268143479dafd556fc6b3bc764 => generated 226 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:24:04.033569 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-162814f0-1277-4a22-aa3d-59b898197cd4 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:24:04.060461 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:24:04.131237 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-162814f0-1277-4a22-aa3d-59b898197cd4 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:24:04.164305 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a34dcfb-fbbe-49b8-a194-77f6b05110cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:24:04.164986 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a34dcfb-fbbe-49b8-a194-77f6b05110cc None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:24:04.165274 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a34dcfb-fbbe-49b8-a194-77f6b05110cc None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:24:04.165555 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a34dcfb-fbbe-49b8-a194-77f6b05110cc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:24:04.166386 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a34dcfb-fbbe-49b8-a194-77f6b05110cc None admin] RBAC: Adding query filter params (group.id=2673ac268143479dafd556fc6b3bc764, role.id=02aba1f7cbdf4d6595e18af7fb5727e0, scope.project.id=ec1ed227a23f453cab64e15036af0c60) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:24:04.166702 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a34dcfb-fbbe-49b8-a194-77f6b05110cc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:24:04.168133 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a34dcfb-fbbe-49b8-a194-77f6b05110cc None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:24:04.174041 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 225/455] 10.222.0.33 () {56 vars in 1376 bytes} [Mon Jul 25 05:24:04 2022] GET /identity/v3/role_assignments?group.id=2673ac268143479dafd556fc6b3bc764&scope.project.id=ec1ed227a23f453cab64e15036af0c60&role.id=02aba1f7cbdf4d6595e18af7fb5727e0 => generated 611 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:24:06.769476 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebf5b1a-4fbd-4b1b-8eb8-8dd51f25ac44 None None] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:24:06.770512 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebf5b1a-4fbd-4b1b-8eb8-8dd51f25ac44 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:24:06.771033 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebf5b1a-4fbd-4b1b-8eb8-8dd51f25ac44 None None] PATH_INFO: `/` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:24:06.772879 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 231/456] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jul 25 05:24:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:24:06.779188 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21882e13-f74a-4d63-a1de-96fb541855a8 None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:24:06.780088 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21882e13-f74a-4d63-a1de-96fb541855a8 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:24:06.780594 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21882e13-f74a-4d63-a1de-96fb541855a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:24:06.805965 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-21882e13-f74a-4d63-a1de-96fb541855a8 None None] MFA Rules not processed for user `1eae540242d94c10b0ad33960a855f08`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:24:06.812801 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-21882e13-f74a-4d63-a1de-96fb541855a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:24:06.883049 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7056e24-39fc-4256-8288-523fb4ce0c8f None None] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:24:06.883686 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7056e24-39fc-4256-8288-523fb4ce0c8f None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:24:06.884173 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7056e24-39fc-4256-8288-523fb4ce0c8f None None] PATH_INFO: `/` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:24:06.886061 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 232/457] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jul 25 05:24:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:24:06.891999 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc2e2a1-ef57-4888-9dbd-b8017856c8a1 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:24:06.892675 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc2e2a1-ef57-4888-9dbd-b8017856c8a1 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:24:06.893201 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc2e2a1-ef57-4888-9dbd-b8017856c8a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:24:06.899428 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 226/458] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jul 25 05:24:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 25 05:24:06.910809 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-ecc2e2a1-ef57-4888-9dbd-b8017856c8a1 None None] MFA Rules not processed for user `1eae540242d94c10b0ad33960a855f08`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:24:06.912260 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-ecc2e2a1-ef57-4888-9dbd-b8017856c8a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:24:06.948831 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21882e13-f74a-4d63-a1de-96fb541855a8 None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:24:06.978956 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:24:07.005165 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 233/459] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jul 25 05:24:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 25 05:24:07.035669 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc2e2a1-ef57-4888-9dbd-b8017856c8a1 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:24:07.125543 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:24:07.131981 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21882e13-f74a-4d63-a1de-96fb541855a8 None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:24:07.180168 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc2e2a1-ef57-4888-9dbd-b8017856c8a1 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:24:07.184163 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0371e569-a497-4740-805e-d05be88ffe1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:24:07.185296 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0371e569-a497-4740-805e-d05be88ffe1d None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:24:07.185793 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0371e569-a497-4740-805e-d05be88ffe1d None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:24:07.186270 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0371e569-a497-4740-805e-d05be88ffe1d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:24:07.201166 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0371e569-a497-4740-805e-d05be88ffe1d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:24:07.203439 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0371e569-a497-4740-805e-d05be88ffe1d None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:24:07.218220 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a288b14c-e862-4edf-bf34-f19c2da165e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:24:07.219245 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a288b14c-e862-4edf-bf34-f19c2da165e8 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:24:07.219703 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a288b14c-e862-4edf-bf34-f19c2da165e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:24:07.220127 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a288b14c-e862-4edf-bf34-f19c2da165e8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:24:07.226408 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application [None req-0371e569-a497-4740-805e-d05be88ffe1d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 05:24:07.226408 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:24:07.226408 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:24:07.226408 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:24:07.226408 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:24:07.226408 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:24:07.226408 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:24:07.226408 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:24:07.226408 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:24:07.226408 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:24:07.226408 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:24:07.226408 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:24:07.226408 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 05:24:07.226408 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 05:24:07.226408 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 05:24:07.227395 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 05:24:07.227395 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:24:07.227395 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:24:07.227395 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:24:07.227395 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:24:07.227395 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:24:07.227395 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:24:07.227395 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:24:07.227395 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application with Lock( Jul 25 05:24:07.227395 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:24:07.227395 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:24:07.227395 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:24:07.227395 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:24:07.227395 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:24:07.227395 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:24:07.227395 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:24:07.227395 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:24:07.228264 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 05:24:07.228264 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 05:24:07.228264 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 05:24:07.228264 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Jul 25 05:24:07.231174 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 227/460] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Jul 25 05:24:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 283 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:24:07.232627 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55771704-f16a-4148-9842-58f80bee4e5a None None] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:24:07.233079 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55771704-f16a-4148-9842-58f80bee4e5a None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:24:07.233489 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55771704-f16a-4148-9842-58f80bee4e5a None None] PATH_INFO: `/` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:24:07.235174 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 228/461] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jul 25 05:24:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:24:07.236382 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55771704-f16a-4148-9842-58f80bee4e5a None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:24:07.242217 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a288b14c-e862-4edf-bf34-f19c2da165e8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:24:07.243369 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a288b14c-e862-4edf-bf34-f19c2da165e8 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:24:07.268419 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application [None req-a288b14c-e862-4edf-bf34-f19c2da165e8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 05:24:07.268419 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:24:07.268419 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:24:07.268419 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:24:07.268419 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:24:07.268419 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:24:07.268419 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:24:07.268419 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:24:07.268419 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:24:07.268419 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:24:07.268419 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:24:07.268419 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:24:07.268419 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 05:24:07.268419 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 05:24:07.268419 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 05:24:07.269496 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 05:24:07.269496 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:24:07.269496 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:24:07.269496 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:24:07.269496 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:24:07.269496 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:24:07.269496 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:24:07.269496 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:24:07.269496 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application with Lock( Jul 25 05:24:07.269496 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:24:07.269496 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:24:07.269496 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:24:07.269496 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:24:07.269496 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:24:07.269496 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:24:07.269496 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:24:07.269496 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:24:07.270346 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 05:24:07.270346 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 05:24:07.270346 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 05:24:07.270346 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Jul 25 05:24:07.270570 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 234/462] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Jul 25 05:24:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:24:07.274062 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8493a19e-3032-4d0f-92ba-2ca51d447b1f None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:24:07.274529 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8493a19e-3032-4d0f-92ba-2ca51d447b1f None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:24:07.274968 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8493a19e-3032-4d0f-92ba-2ca51d447b1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:24:07.296553 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-8493a19e-3032-4d0f-92ba-2ca51d447b1f None None] MFA Rules not processed for user `1eae540242d94c10b0ad33960a855f08`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:24:07.298199 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-8493a19e-3032-4d0f-92ba-2ca51d447b1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:24:07.331656 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:24:07.366147 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55771704-f16a-4148-9842-58f80bee4e5a None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:24:07.375825 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 235/463] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jul 25 05:24:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 05:24:07.377040 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8493a19e-3032-4d0f-92ba-2ca51d447b1f None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:24:07.408837 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d62fb7a-78ae-4bbf-8ef7-9806c24bc842 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:24:07.408837 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d62fb7a-78ae-4bbf-8ef7-9806c24bc842 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:24:07.408837 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d62fb7a-78ae-4bbf-8ef7-9806c24bc842 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:24:07.408837 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d62fb7a-78ae-4bbf-8ef7-9806c24bc842 None admin] PATH_INFO: `/v3/domains` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:24:07.409245 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d62fb7a-78ae-4bbf-8ef7-9806c24bc842 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:24:07.409245 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d62fb7a-78ae-4bbf-8ef7-9806c24bc842 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:24:07.409245 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d62fb7a-78ae-4bbf-8ef7-9806c24bc842 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:24:07.427190 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 229/464] 10.222.0.33 () {56 vars in 1109 bytes} [Mon Jul 25 05:24:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 05:24:07.428302 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d62fb7a-78ae-4bbf-8ef7-9806c24bc842 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:24:07.432851 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:24:07.514171 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8493a19e-3032-4d0f-92ba-2ca51d447b1f None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:24:07.538370 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d62fb7a-78ae-4bbf-8ef7-9806c24bc842 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:24:07.553049 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a4cf0e-ce1d-4d02-9219-b6d3fea681dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:24:07.553709 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a4cf0e-ce1d-4d02-9219-b6d3fea681dd None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:24:07.554003 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a4cf0e-ce1d-4d02-9219-b6d3fea681dd None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:24:07.554285 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a4cf0e-ce1d-4d02-9219-b6d3fea681dd None admin] PATH_INFO: `/v3/domains` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:24:07.555189 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3a4cf0e-ce1d-4d02-9219-b6d3fea681dd None admin] RBAC: Adding query filter params (name=Default) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:24:07.555506 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3a4cf0e-ce1d-4d02-9219-b6d3fea681dd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:24:07.556929 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3a4cf0e-ce1d-4d02-9219-b6d3fea681dd None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:24:07.573239 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 236/465] 10.222.0.33 () {56 vars in 1109 bytes} [Mon Jul 25 05:24:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:24:07.574419 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a4cf0e-ce1d-4d02-9219-b6d3fea681dd None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:24:07.575773 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:24:07.580272 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e530f7b2-24d4-40b2-9710-5b92772bf4b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:24:07.581369 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e530f7b2-24d4-40b2-9710-5b92772bf4b1 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:24:07.581903 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e530f7b2-24d4-40b2-9710-5b92772bf4b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:24:07.582395 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e530f7b2-24d4-40b2-9710-5b92772bf4b1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:24:07.596830 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e530f7b2-24d4-40b2-9710-5b92772bf4b1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:24:07.598673 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e530f7b2-24d4-40b2-9710-5b92772bf4b1 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:24:07.618521 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application [None req-e530f7b2-24d4-40b2-9710-5b92772bf4b1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 05:24:07.618521 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:24:07.618521 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:24:07.618521 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:24:07.618521 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:24:07.618521 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:24:07.618521 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:24:07.618521 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:24:07.618521 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:24:07.618521 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:24:07.618521 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:24:07.618521 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:24:07.618521 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 05:24:07.618521 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 05:24:07.618521 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 05:24:07.619458 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 05:24:07.619458 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:24:07.619458 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:24:07.619458 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:24:07.619458 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:24:07.619458 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:24:07.619458 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:24:07.619458 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:24:07.619458 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application with Lock( Jul 25 05:24:07.619458 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:24:07.619458 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:24:07.619458 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:24:07.619458 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:24:07.619458 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:24:07.619458 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:24:07.619458 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:24:07.619458 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:24:07.620338 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 05:24:07.620338 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 05:24:07.620338 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 05:24:07.620338 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Jul 25 05:24:07.621243 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 230/466] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Jul 25 05:24:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:24:07.622337 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e530f7b2-24d4-40b2-9710-5b92772bf4b1 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:24:07.686118 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:24:07.696421 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a4cf0e-ce1d-4d02-9219-b6d3fea681dd None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:24:07.729423 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e530f7b2-24d4-40b2-9710-5b92772bf4b1 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:24:07.730940 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f93fd2-9adb-471c-a027-805ee974b8b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:24:07.732336 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f93fd2-9adb-471c-a027-805ee974b8b7 None admin] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:24:07.732619 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f93fd2-9adb-471c-a027-805ee974b8b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:24:07.732928 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f93fd2-9adb-471c-a027-805ee974b8b7 None admin] PATH_INFO: `/v3/users` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:24:07.733674 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41f93fd2-9adb-471c-a027-805ee974b8b7 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:24:07.735120 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41f93fd2-9adb-471c-a027-805ee974b8b7 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:24:07.755076 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 237/467] 10.222.0.33 () {60 vars in 1132 bytes} [Mon Jul 25 05:24:07 2022] POST /identity/v3/users => generated 246 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 05:24:07.756255 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d92b26a-d62c-4b30-a2c6-f25ecf9a6ba8 None None] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:24:07.756550 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d92b26a-d62c-4b30-a2c6-f25ecf9a6ba8 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:24:07.756861 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d92b26a-d62c-4b30-a2c6-f25ecf9a6ba8 None None] PATH_INFO: `/` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:24:07.763208 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 238/468] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jul 25 05:24:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:24:07.763208 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d92b26a-d62c-4b30-a2c6-f25ecf9a6ba8 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:24:07.782174 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf130796-5df8-46a3-860a-29ef1a596bbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:24:07.783842 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf130796-5df8-46a3-860a-29ef1a596bbe None admin] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:24:07.784307 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf130796-5df8-46a3-860a-29ef1a596bbe None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:24:07.785049 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf130796-5df8-46a3-860a-29ef1a596bbe None admin] PATH_INFO: `/v3/users` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:24:07.786458 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf130796-5df8-46a3-860a-29ef1a596bbe None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:24:07.788186 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf130796-5df8-46a3-860a-29ef1a596bbe None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:24:07.829588 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 231/469] 10.222.0.33 () {60 vars in 1132 bytes} [Mon Jul 25 05:24:07 2022] POST /identity/v3/users => generated 248 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 05:24:07.830931 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e52acb-abb3-4f6c-97be-5b9b61ce9bae None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:24:07.831309 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e52acb-abb3-4f6c-97be-5b9b61ce9bae None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:24:07.831665 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e52acb-abb3-4f6c-97be-5b9b61ce9bae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:24:07.874239 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-98e52acb-abb3-4f6c-97be-5b9b61ce9bae None None] MFA Rules not processed for user `1eae540242d94c10b0ad33960a855f08`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:24:07.875856 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-98e52acb-abb3-4f6c-97be-5b9b61ce9bae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:24:07.900549 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d92b26a-d62c-4b30-a2c6-f25ecf9a6ba8 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:24:07.950472 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:24:07.955028 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc598e3-29e4-42b0-9b1f-092518a139a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:24:07.956361 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc598e3-29e4-42b0-9b1f-092518a139a8 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:24:07.957057 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc598e3-29e4-42b0-9b1f-092518a139a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:24:07.957461 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc598e3-29e4-42b0-9b1f-092518a139a8 None admin] PATH_INFO: `/v3/domains` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:24:07.958397 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccc598e3-29e4-42b0-9b1f-092518a139a8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:24:07.958886 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccc598e3-29e4-42b0-9b1f-092518a139a8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:24:07.961429 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccc598e3-29e4-42b0-9b1f-092518a139a8 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:24:07.988610 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 239/470] 10.222.0.33 () {56 vars in 1109 bytes} [Mon Jul 25 05:24:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:24:07.990866 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc598e3-29e4-42b0-9b1f-092518a139a8 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:24:08.006813 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:24:08.012963 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 232/471] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jul 25 05:24:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 05:24:08.039743 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e52acb-abb3-4f6c-97be-5b9b61ce9bae None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:24:08.132883 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc598e3-29e4-42b0-9b1f-092518a139a8 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:24:08.171822 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e52acb-abb3-4f6c-97be-5b9b61ce9bae None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:24:08.178513 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b68be77-b378-4164-85ac-bb882a42c996 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:24:08.179229 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b68be77-b378-4164-85ac-bb882a42c996 None admin] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:24:08.179519 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b68be77-b378-4164-85ac-bb882a42c996 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:24:08.179803 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b68be77-b378-4164-85ac-bb882a42c996 None admin] PATH_INFO: `/v3/users` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:24:08.180532 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b68be77-b378-4164-85ac-bb882a42c996 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:24:08.182231 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b68be77-b378-4164-85ac-bb882a42c996 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:24:08.216824 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 240/472] 10.222.0.33 () {60 vars in 1132 bytes} [Mon Jul 25 05:24:07 2022] POST /identity/v3/users => generated 248 bytes in 220 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 05:24:08.221015 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff1f288-a290-46b8-8305-ae42a0d8aade None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:24:08.221892 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff1f288-a290-46b8-8305-ae42a0d8aade None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:24:08.222272 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff1f288-a290-46b8-8305-ae42a0d8aade None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:24:08.222557 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff1f288-a290-46b8-8305-ae42a0d8aade None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:24:08.235994 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ff1f288-a290-46b8-8305-ae42a0d8aade None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:24:08.237545 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ff1f288-a290-46b8-8305-ae42a0d8aade None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:24:08.257994 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application [None req-0ff1f288-a290-46b8-8305-ae42a0d8aade None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 05:24:08.257994 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:24:08.257994 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:24:08.257994 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:24:08.257994 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:24:08.257994 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:24:08.257994 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:24:08.257994 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:24:08.257994 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:24:08.257994 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:24:08.257994 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:24:08.257994 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:24:08.257994 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 05:24:08.257994 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 05:24:08.257994 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 05:24:08.258986 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 05:24:08.258986 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:24:08.258986 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:24:08.258986 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:24:08.258986 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:24:08.258986 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:24:08.258986 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:24:08.258986 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:24:08.258986 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application with Lock( Jul 25 05:24:08.258986 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:24:08.258986 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:24:08.258986 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:24:08.258986 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:24:08.258986 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:24:08.258986 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:24:08.258986 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:24:08.258986 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:24:08.260483 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 05:24:08.260483 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 05:24:08.260483 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 05:24:08.260483 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Jul 25 05:24:08.260483 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 233/473] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Jul 25 05:24:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 25 05:24:08.265455 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b68be77-b378-4164-85ac-bb882a42c996 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:24:08.293126 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30fb078e-7975-4cdc-b557-48094bd88610 None None] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:24:08.293448 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30fb078e-7975-4cdc-b557-48094bd88610 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:24:08.294289 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30fb078e-7975-4cdc-b557-48094bd88610 None None] PATH_INFO: `/` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:24:08.296015 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 234/474] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jul 25 05:24:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 25 05:24:08.302275 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65cf71f-5bfe-4249-8d40-d470d5f394a6 None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:24:08.302588 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65cf71f-5bfe-4249-8d40-d470d5f394a6 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:24:08.302850 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65cf71f-5bfe-4249-8d40-d470d5f394a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:24:08.322026 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-b65cf71f-5bfe-4249-8d40-d470d5f394a6 None None] MFA Rules not processed for user `1eae540242d94c10b0ad33960a855f08`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:24:08.323253 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-b65cf71f-5bfe-4249-8d40-d470d5f394a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:24:08.343770 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:24:08.406217 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 235/475] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jul 25 05:24:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 05:24:08.408540 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:24:08.429931 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b65cf71f-5bfe-4249-8d40-d470d5f394a6 None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:24:08.461285 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b68be77-b378-4164-85ac-bb882a42c996 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:24:08.508921 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe68c77-c987-42c0-830a-8a7d5d1c798b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:24:08.509729 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe68c77-c987-42c0-830a-8a7d5d1c798b None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:24:08.510017 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe68c77-c987-42c0-830a-8a7d5d1c798b None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:24:08.510296 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe68c77-c987-42c0-830a-8a7d5d1c798b None admin] PATH_INFO: `/v3/domains` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:24:08.511132 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afe68c77-c987-42c0-830a-8a7d5d1c798b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:24:08.511441 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afe68c77-c987-42c0-830a-8a7d5d1c798b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:24:08.514537 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afe68c77-c987-42c0-830a-8a7d5d1c798b None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:24:08.528960 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 241/476] 10.222.0.33 () {56 vars in 1109 bytes} [Mon Jul 25 05:24:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:24:08.536304 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe68c77-c987-42c0-830a-8a7d5d1c798b None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:24:08.572574 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b65cf71f-5bfe-4249-8d40-d470d5f394a6 None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:24:08.620486 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e492bc-c584-45dc-9e6a-138e58d2c34e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:24:08.621583 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e492bc-c584-45dc-9e6a-138e58d2c34e None admin] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:24:08.622084 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e492bc-c584-45dc-9e6a-138e58d2c34e None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:24:08.622398 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e492bc-c584-45dc-9e6a-138e58d2c34e None admin] PATH_INFO: `/v3/roles` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:24:08.623166 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39e492bc-c584-45dc-9e6a-138e58d2c34e None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:24:08.625875 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39e492bc-c584-45dc-9e6a-138e58d2c34e None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:24:08.635968 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.sql.core [None req-39e492bc-c584-45dc-9e6a-138e58d2c34e None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jul 25 05:24:08.635968 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Jul 25 05:24:08.635968 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [parameters: {'id': 'c016e0084ad440a0ba308a143a49fb90', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Jul 25 05:24:08.635968 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80248) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jul 25 05:24:08.660275 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application [None req-39e492bc-c584-45dc-9e6a-138e58d2c34e None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Jul 25 05:24:08.660275 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:24:08.660275 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 25 05:24:08.660275 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 25 05:24:08.660275 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 25 05:24:08.660275 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 25 05:24:08.660275 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 25 05:24:08.660275 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application result = self._query(query) Jul 25 05:24:08.660275 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 25 05:24:08.660275 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application conn.query(q) Jul 25 05:24:08.660275 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 25 05:24:08.660275 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 25 05:24:08.660275 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 25 05:24:08.660275 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application result.read() Jul 25 05:24:08.660275 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 25 05:24:08.661236 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 25 05:24:08.661236 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 25 05:24:08.661236 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 25 05:24:08.661236 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 25 05:24:08.661236 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 25 05:24:08.661236 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 25 05:24:08.661236 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 25 05:24:08.661236 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jul 25 05:24:08.661236 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Jul 25 05:24:08.661236 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jul 25 05:24:08.661236 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Jul 25 05:24:08.661236 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:24:08.661236 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jul 25 05:24:08.661236 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jul 25 05:24:08.661236 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Jul 25 05:24:08.661236 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return ref.to_dict() Jul 25 05:24:08.661236 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 25 05:24:08.661236 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application next(self.gen) Jul 25 05:24:08.662185 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jul 25 05:24:08.662185 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application yield resource Jul 25 05:24:08.662185 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 25 05:24:08.662185 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application next(self.gen) Jul 25 05:24:08.662185 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jul 25 05:24:08.662185 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application self.session.rollback() Jul 25 05:24:08.662185 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jul 25 05:24:08.662185 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application self.force_reraise() Jul 25 05:24:08.662185 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jul 25 05:24:08.662185 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application raise self.value Jul 25 05:24:08.662185 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jul 25 05:24:08.662185 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jul 25 05:24:08.662185 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jul 25 05:24:08.662185 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application session.commit() Jul 25 05:24:08.662185 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jul 25 05:24:08.662185 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jul 25 05:24:08.662185 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jul 25 05:24:08.663058 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application self._prepare_impl() Jul 25 05:24:08.663058 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jul 25 05:24:08.663058 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application self.session.flush() Jul 25 05:24:08.663058 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jul 25 05:24:08.663058 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application self._flush(objects) Jul 25 05:24:08.663058 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jul 25 05:24:08.663058 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jul 25 05:24:08.663058 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jul 25 05:24:08.663058 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application compat.raise_( Jul 25 05:24:08.663058 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 25 05:24:08.663058 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application raise exception Jul 25 05:24:08.663058 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jul 25 05:24:08.663058 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application flush_context.execute() Jul 25 05:24:08.663058 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jul 25 05:24:08.663058 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application rec.execute(self) Jul 25 05:24:08.663058 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jul 25 05:24:08.663058 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jul 25 05:24:08.663939 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jul 25 05:24:08.663939 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application _emit_insert_statements( Jul 25 05:24:08.663939 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Jul 25 05:24:08.663939 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application c = connection._execute_20( Jul 25 05:24:08.663939 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jul 25 05:24:08.663939 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jul 25 05:24:08.663939 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Jul 25 05:24:08.663939 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jul 25 05:24:08.663939 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jul 25 05:24:08.663939 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application ret = self._execute_context( Jul 25 05:24:08.663939 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jul 25 05:24:08.663939 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jul 25 05:24:08.663939 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jul 25 05:24:08.663939 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jul 25 05:24:08.663939 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 25 05:24:08.663939 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application raise exception Jul 25 05:24:08.664772 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 25 05:24:08.664772 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 25 05:24:08.664772 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 25 05:24:08.664772 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 25 05:24:08.664772 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 25 05:24:08.664772 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application result = self._query(query) Jul 25 05:24:08.664772 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 25 05:24:08.664772 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application conn.query(q) Jul 25 05:24:08.664772 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 25 05:24:08.664772 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 25 05:24:08.664772 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 25 05:24:08.664772 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application result.read() Jul 25 05:24:08.664772 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 25 05:24:08.664772 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 25 05:24:08.664772 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 25 05:24:08.664772 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 25 05:24:08.673954 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 25 05:24:08.673954 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 25 05:24:08.673954 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 25 05:24:08.673954 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 25 05:24:08.673954 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jul 25 05:24:08.673954 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Jul 25 05:24:08.673954 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application [parameters: {'id': 'c016e0084ad440a0ba308a143a49fb90', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Jul 25 05:24:08.673954 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jul 25 05:24:08.673954 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Jul 25 05:24:08.673954 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 25 05:24:08.673954 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Jul 25 05:24:08.673954 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:24:08.673954 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:24:08.673954 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:24:08.673954 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:24:08.673954 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:24:08.676996 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:24:08.676996 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:24:08.676996 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:24:08.676996 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:24:08.676996 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:24:08.676996 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:24:08.676996 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Jul 25 05:24:08.676996 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Jul 25 05:24:08.676996 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:24:08.676996 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:24:08.676996 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Jul 25 05:24:08.676996 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Jul 25 05:24:08.676996 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jul 25 05:24:08.676996 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jul 25 05:24:08.676996 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Jul 25 05:24:08.676996 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Jul 25 05:24:08.679665 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe68c77-c987-42c0-830a-8a7d5d1c798b None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:24:08.679826 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 236/477] 10.222.0.33 () {60 vars in 1131 bytes} [Mon Jul 25 05:24:08 2022] POST /identity/v3/roles => generated 144 bytes in 237 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:24:08.685270 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e492bc-c584-45dc-9e6a-138e58d2c34e None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:24:08.697054 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:24:08.712315 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb16a11-e2a3-4157-ae97-284bacbcb1e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:24:08.713199 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb16a11-e2a3-4157-ae97-284bacbcb1e9 None admin] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:24:08.713488 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb16a11-e2a3-4157-ae97-284bacbcb1e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:24:08.713828 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb16a11-e2a3-4157-ae97-284bacbcb1e9 None admin] PATH_INFO: `/v3/users` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:24:08.714611 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fb16a11-e2a3-4157-ae97-284bacbcb1e9 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:24:08.716270 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fb16a11-e2a3-4157-ae97-284bacbcb1e9 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:24:08.737171 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:24:08.739871 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 242/478] 10.222.0.33 () {60 vars in 1132 bytes} [Mon Jul 25 05:24:08 2022] POST /identity/v3/users => generated 249 bytes in 204 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 05:24:08.867096 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e492bc-c584-45dc-9e6a-138e58d2c34e None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:24:08.917499 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-939107d0-6bbf-4df2-96e2-25897c47c979 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:24:08.918757 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939107d0-6bbf-4df2-96e2-25897c47c979 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:24:08.919402 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939107d0-6bbf-4df2-96e2-25897c47c979 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:24:08.919898 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939107d0-6bbf-4df2-96e2-25897c47c979 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:24:08.938640 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-939107d0-6bbf-4df2-96e2-25897c47c979 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 05:24:08.947326 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application [None req-939107d0-6bbf-4df2-96e2-25897c47c979 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 25 05:24:08.947326 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:24:08.947326 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:24:08.947326 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:24:08.947326 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:24:08.947326 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:24:08.947326 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:24:08.947326 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:24:08.947326 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:24:08.947326 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:24:08.947326 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:24:08.947326 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:24:08.947326 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 05:24:08.947326 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 05:24:08.947326 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 05:24:08.947326 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 05:24:08.948284 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 05:24:08.948284 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 05:24:08.948284 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 05:24:08.948284 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 05:24:08.948284 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:24:08.948284 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:24:08.948284 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:24:08.948284 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:24:08.948284 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:24:08.948284 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:24:08.948284 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:24:08.948284 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application with Lock( Jul 25 05:24:08.948284 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:24:08.948284 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:24:08.948284 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:24:08.948284 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:24:08.953565 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:24:08.953565 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:24:08.953565 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:24:08.953565 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:24:08.953565 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 05:24:08.953565 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 05:24:08.953565 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 05:24:08.953565 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 05:24:08.953565 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 05:24:08.953565 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 05:24:08.953565 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 25 05:24:08.953565 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Jul 25 05:24:08.956113 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 237/479] 10.222.0.33 () {56 vars in 1114 bytes} [Mon Jul 25 05:24:08 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 272 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 25 05:24:08.962038 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb16a11-e2a3-4157-ae97-284bacbcb1e9 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:24:09.078371 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb16a11-e2a3-4157-ae97-284bacbcb1e9 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:24:09.118523 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ff7b92-0f41-4247-9e22-7a15edb91bcb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:24:09.119202 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ff7b92-0f41-4247-9e22-7a15edb91bcb None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:24:09.119491 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ff7b92-0f41-4247-9e22-7a15edb91bcb None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:24:09.119772 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ff7b92-0f41-4247-9e22-7a15edb91bcb None admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:24:09.120534 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8ff7b92-0f41-4247-9e22-7a15edb91bcb None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:24:09.120880 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8ff7b92-0f41-4247-9e22-7a15edb91bcb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:24:09.122436 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8ff7b92-0f41-4247-9e22-7a15edb91bcb None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:24:09.131564 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 243/480] 10.222.0.33 () {56 vars in 1111 bytes} [Mon Jul 25 05:24:08 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:24:10.409335 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f6a73c-d501-4c83-854d-b3d123471600 None None] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:24:10.409859 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f6a73c-d501-4c83-854d-b3d123471600 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:24:10.409859 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f6a73c-d501-4c83-854d-b3d123471600 None None] PATH_INFO: `/` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:24:10.411535 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 238/481] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jul 25 05:24:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:24:10.416966 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ca8058-a748-4455-b3af-4f91941f4512 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:24:10.417408 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ca8058-a748-4455-b3af-4f91941f4512 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:24:10.417581 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ca8058-a748-4455-b3af-4f91941f4512 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:24:10.442314 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-72ca8058-a748-4455-b3af-4f91941f4512 None None] MFA Rules not processed for user `1eae540242d94c10b0ad33960a855f08`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:24:10.443524 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-72ca8058-a748-4455-b3af-4f91941f4512 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:24:10.478453 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4fd461-4c98-4c71-8236-7880399f2c3a None None] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:24:10.479103 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4fd461-4c98-4c71-8236-7880399f2c3a None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:24:10.479641 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4fd461-4c98-4c71-8236-7880399f2c3a None None] PATH_INFO: `/` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:24:10.481760 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 239/482] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jul 25 05:24:10 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 25 05:24:10.487947 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64fe1e79-efa1-41d7-b2a4-48f14f07ce8e None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:24:10.488674 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64fe1e79-efa1-41d7-b2a4-48f14f07ce8e None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:24:10.489611 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64fe1e79-efa1-41d7-b2a4-48f14f07ce8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:24:10.500341 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:24:10.517674 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-64fe1e79-efa1-41d7-b2a4-48f14f07ce8e None None] MFA Rules not processed for user `1eae540242d94c10b0ad33960a855f08`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:24:10.518900 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-64fe1e79-efa1-41d7-b2a4-48f14f07ce8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:24:10.556527 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 244/483] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jul 25 05:24:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 25 05:24:10.578355 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ca8058-a748-4455-b3af-4f91941f4512 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:24:10.654920 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 240/484] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jul 25 05:24:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 05:24:10.707572 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64fe1e79-efa1-41d7-b2a4-48f14f07ce8e None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:24:10.715518 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:24:10.725636 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ca8058-a748-4455-b3af-4f91941f4512 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:24:10.776688 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4802e0cc-d5cd-40c2-99ca-aa2bf4af5fd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:24:10.777939 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4802e0cc-d5cd-40c2-99ca-aa2bf4af5fd1 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:24:10.778182 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4802e0cc-d5cd-40c2-99ca-aa2bf4af5fd1 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:24:10.778461 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4802e0cc-d5cd-40c2-99ca-aa2bf4af5fd1 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:24:10.789509 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4802e0cc-d5cd-40c2-99ca-aa2bf4af5fd1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 05:24:10.798984 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application [None req-4802e0cc-d5cd-40c2-99ca-aa2bf4af5fd1 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 25 05:24:10.798984 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:24:10.798984 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:24:10.798984 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:24:10.798984 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:24:10.798984 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:24:10.798984 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:24:10.798984 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:24:10.798984 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:24:10.798984 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:24:10.798984 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:24:10.798984 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:24:10.798984 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 05:24:10.798984 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 05:24:10.798984 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 05:24:10.798984 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 05:24:10.799914 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 05:24:10.799914 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 05:24:10.799914 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 05:24:10.799914 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 05:24:10.799914 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:24:10.799914 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:24:10.799914 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:24:10.799914 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:24:10.799914 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:24:10.799914 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:24:10.799914 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:24:10.799914 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application with Lock( Jul 25 05:24:10.799914 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:24:10.799914 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:24:10.799914 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:24:10.799914 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:24:10.801186 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:24:10.801186 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:24:10.801186 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:24:10.801186 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:24:10.801186 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 05:24:10.801186 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 05:24:10.801186 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 05:24:10.801186 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 05:24:10.801186 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 05:24:10.801186 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 05:24:10.801186 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 25 05:24:10.801186 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Jul 25 05:24:10.801186 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 245/485] 10.222.0.33 () {56 vars in 1102 bytes} [Mon Jul 25 05:24:10 2022] GET /identity/v3/roles/service => generated 85 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:24:10.809491 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4802e0cc-d5cd-40c2-99ca-aa2bf4af5fd1 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:24:10.852487 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:24:10.890627 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64fe1e79-efa1-41d7-b2a4-48f14f07ce8e None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:24:10.937390 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11017fdd-50ef-4160-b9a1-2117a13fcc3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:24:10.938212 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11017fdd-50ef-4160-b9a1-2117a13fcc3a None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:24:10.938506 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11017fdd-50ef-4160-b9a1-2117a13fcc3a None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:24:10.939048 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11017fdd-50ef-4160-b9a1-2117a13fcc3a None admin] PATH_INFO: `/v3/roles/service` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:24:10.950477 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4802e0cc-d5cd-40c2-99ca-aa2bf4af5fd1 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:24:10.951822 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11017fdd-50ef-4160-b9a1-2117a13fcc3a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 05:24:10.959647 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application [None req-11017fdd-50ef-4160-b9a1-2117a13fcc3a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 25 05:24:10.959647 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:24:10.959647 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:24:10.959647 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:24:10.959647 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:24:10.959647 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:24:10.959647 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:24:10.959647 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:24:10.959647 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:24:10.959647 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:24:10.959647 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:24:10.959647 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:24:10.959647 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 05:24:10.959647 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 05:24:10.959647 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 05:24:10.959647 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 05:24:10.960535 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 05:24:10.960535 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 05:24:10.960535 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 05:24:10.960535 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 05:24:10.960535 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:24:10.960535 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:24:10.960535 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:24:10.960535 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:24:10.960535 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:24:10.960535 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:24:10.960535 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:24:10.960535 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application with Lock( Jul 25 05:24:10.960535 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:24:10.960535 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:24:10.960535 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:24:10.960535 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:24:10.961364 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:24:10.961364 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:24:10.961364 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:24:10.961364 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:24:10.961364 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 05:24:10.961364 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 05:24:10.961364 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 05:24:10.961364 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 05:24:10.961364 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 05:24:10.961364 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 05:24:10.961364 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 25 05:24:10.961364 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Jul 25 05:24:10.961364 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 241/486] 10.222.0.33 () {56 vars in 1102 bytes} [Mon Jul 25 05:24:10 2022] GET /identity/v3/roles/service => generated 85 bytes in 254 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 25 05:24:10.962574 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e21107f-c5a7-44e3-b118-b1c3ea6f718d None None] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:24:10.962865 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e21107f-c5a7-44e3-b118-b1c3ea6f718d None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:24:10.963149 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e21107f-c5a7-44e3-b118-b1c3ea6f718d None None] PATH_INFO: `/` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:24:10.964423 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 242/487] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jul 25 05:24:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:24:10.969702 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e21107f-c5a7-44e3-b118-b1c3ea6f718d None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:24:10.990545 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd63ad3-1c36-4de5-b38f-d1aeb4ad1b2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:24:10.991539 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd63ad3-1c36-4de5-b38f-d1aeb4ad1b2b None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:24:10.992002 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd63ad3-1c36-4de5-b38f-d1aeb4ad1b2b None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:24:10.992466 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd63ad3-1c36-4de5-b38f-d1aeb4ad1b2b None admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:24:10.993430 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efd63ad3-1c36-4de5-b38f-d1aeb4ad1b2b None admin] RBAC: Adding query filter params (name=service) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:24:10.993933 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efd63ad3-1c36-4de5-b38f-d1aeb4ad1b2b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:24:10.995475 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efd63ad3-1c36-4de5-b38f-d1aeb4ad1b2b None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:24:11.005084 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 246/488] 10.222.0.33 () {56 vars in 1103 bytes} [Mon Jul 25 05:24:10 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:24:11.006792 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee307de5-07b9-43af-87aa-939a197d0cdc None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:24:11.007254 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee307de5-07b9-43af-87aa-939a197d0cdc None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:24:11.007679 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee307de5-07b9-43af-87aa-939a197d0cdc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:24:11.024308 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-ee307de5-07b9-43af-87aa-939a197d0cdc None None] MFA Rules not processed for user `1eae540242d94c10b0ad33960a855f08`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:24:11.025732 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-ee307de5-07b9-43af-87aa-939a197d0cdc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:24:11.042189 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:24:11.082130 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e21107f-c5a7-44e3-b118-b1c3ea6f718d None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:24:11.097848 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 247/489] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jul 25 05:24:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 05:24:11.098913 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee307de5-07b9-43af-87aa-939a197d0cdc None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:24:11.124682 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:24:11.142092 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f2fece-b0e3-4e70-b858-b97cd6ef58b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:24:11.142931 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f2fece-b0e3-4e70-b858-b97cd6ef58b8 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:24:11.143328 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f2fece-b0e3-4e70-b858-b97cd6ef58b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:24:11.144001 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f2fece-b0e3-4e70-b858-b97cd6ef58b8 None admin] PATH_INFO: `/v3/roles` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:24:11.144569 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35f2fece-b0e3-4e70-b858-b97cd6ef58b8 None admin] RBAC: Adding query filter params (name=service) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:24:11.145105 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35f2fece-b0e3-4e70-b858-b97cd6ef58b8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:24:11.146692 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35f2fece-b0e3-4e70-b858-b97cd6ef58b8 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:24:11.159450 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 243/490] 10.222.0.33 () {56 vars in 1103 bytes} [Mon Jul 25 05:24:10 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:24:11.160430 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f2fece-b0e3-4e70-b858-b97cd6ef58b8 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:24:11.207588 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee307de5-07b9-43af-87aa-939a197d0cdc None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:24:11.242670 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a04e68e1-3079-4870-84fa-711d9fecd58b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:24:11.243709 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04e68e1-3079-4870-84fa-711d9fecd58b None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:24:11.244139 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04e68e1-3079-4870-84fa-711d9fecd58b None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:24:11.244624 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04e68e1-3079-4870-84fa-711d9fecd58b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:24:11.256102 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a04e68e1-3079-4870-84fa-711d9fecd58b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:24:11.257908 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a04e68e1-3079-4870-84fa-711d9fecd58b None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:24:11.270555 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f2fece-b0e3-4e70-b858-b97cd6ef58b8 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:24:11.277171 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application [None req-a04e68e1-3079-4870-84fa-711d9fecd58b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 05:24:11.277171 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:24:11.277171 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:24:11.277171 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:24:11.277171 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:24:11.277171 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:24:11.277171 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:24:11.277171 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:24:11.277171 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:24:11.277171 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:24:11.277171 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:24:11.277171 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:24:11.277171 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 05:24:11.277171 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 05:24:11.277171 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 05:24:11.278032 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 05:24:11.278032 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:24:11.278032 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:24:11.278032 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:24:11.278032 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:24:11.278032 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:24:11.278032 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:24:11.278032 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:24:11.278032 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application with Lock( Jul 25 05:24:11.278032 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:24:11.278032 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:24:11.278032 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:24:11.278032 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:24:11.278032 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:24:11.278032 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:24:11.278032 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:24:11.278032 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:24:11.278982 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 05:24:11.278982 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 05:24:11.278982 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 05:24:11.278982 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Jul 25 05:24:11.281197 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 248/491] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Jul 25 05:24:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:24:11.282215 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a04e68e1-3079-4870-84fa-711d9fecd58b None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:24:11.298675 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:24:11.308338 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba441f0-44cf-4117-a61c-dcacfac1afa9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:24:11.309995 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba441f0-44cf-4117-a61c-dcacfac1afa9 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:24:11.310435 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba441f0-44cf-4117-a61c-dcacfac1afa9 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:24:11.310932 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba441f0-44cf-4117-a61c-dcacfac1afa9 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:24:11.322593 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eba441f0-44cf-4117-a61c-dcacfac1afa9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 05:24:11.331848 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application [None req-eba441f0-44cf-4117-a61c-dcacfac1afa9 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 25 05:24:11.331848 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:24:11.331848 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:24:11.331848 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:24:11.331848 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:24:11.331848 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:24:11.331848 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:24:11.331848 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:24:11.331848 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:24:11.331848 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:24:11.331848 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:24:11.331848 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:24:11.331848 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 05:24:11.331848 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 05:24:11.331848 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 05:24:11.331848 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 05:24:11.332748 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 05:24:11.332748 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 05:24:11.332748 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 05:24:11.332748 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 05:24:11.332748 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:24:11.332748 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:24:11.332748 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:24:11.332748 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:24:11.332748 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:24:11.332748 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:24:11.332748 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:24:11.332748 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application with Lock( Jul 25 05:24:11.332748 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:24:11.332748 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:24:11.332748 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:24:11.332748 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:24:11.333630 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:24:11.333630 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:24:11.333630 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:24:11.333630 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:24:11.333630 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 05:24:11.333630 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 05:24:11.333630 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 05:24:11.333630 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 05:24:11.333630 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 05:24:11.333630 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 05:24:11.333630 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 25 05:24:11.333630 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Jul 25 05:24:11.341035 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 244/492] 10.222.0.33 () {56 vars in 1102 bytes} [Mon Jul 25 05:24:11 2022] GET /identity/v3/roles/service => generated 85 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:24:11.342247 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba441f0-44cf-4117-a61c-dcacfac1afa9 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:24:11.366435 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:24:11.391733 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a04e68e1-3079-4870-84fa-711d9fecd58b None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:24:11.428444 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-637a0a97-b6d6-4c32-9570-dc2f9e207baf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:24:11.429396 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637a0a97-b6d6-4c32-9570-dc2f9e207baf None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:24:11.429908 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637a0a97-b6d6-4c32-9570-dc2f9e207baf None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:24:11.430044 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637a0a97-b6d6-4c32-9570-dc2f9e207baf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:24:11.442377 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-637a0a97-b6d6-4c32-9570-dc2f9e207baf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:24:11.443905 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-637a0a97-b6d6-4c32-9570-dc2f9e207baf None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:24:11.454344 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba441f0-44cf-4117-a61c-dcacfac1afa9 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:24:11.462194 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application [None req-637a0a97-b6d6-4c32-9570-dc2f9e207baf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 05:24:11.462194 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:24:11.462194 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:24:11.462194 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:24:11.462194 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:24:11.462194 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:24:11.462194 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:24:11.462194 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:24:11.462194 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:24:11.462194 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:24:11.462194 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:24:11.462194 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:24:11.462194 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 05:24:11.462194 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 05:24:11.462194 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 05:24:11.463033 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 05:24:11.463033 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:24:11.463033 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:24:11.463033 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:24:11.463033 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:24:11.463033 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:24:11.463033 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:24:11.463033 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:24:11.463033 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application with Lock( Jul 25 05:24:11.463033 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:24:11.463033 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:24:11.463033 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:24:11.463033 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:24:11.463033 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:24:11.463033 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:24:11.463033 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:24:11.463033 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:24:11.464985 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 05:24:11.464985 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 05:24:11.464985 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 05:24:11.464985 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Jul 25 05:24:11.464985 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 249/493] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Jul 25 05:24:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:24:11.464985 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-637a0a97-b6d6-4c32-9570-dc2f9e207baf None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:24:11.494286 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-066c1b77-51fc-485d-854f-212b84ac731b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:24:11.494963 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066c1b77-51fc-485d-854f-212b84ac731b None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:24:11.495256 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066c1b77-51fc-485d-854f-212b84ac731b None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:24:11.495541 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066c1b77-51fc-485d-854f-212b84ac731b None admin] PATH_INFO: `/v3/domains` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:24:11.496302 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-066c1b77-51fc-485d-854f-212b84ac731b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:24:11.497252 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-066c1b77-51fc-485d-854f-212b84ac731b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:24:11.498643 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-066c1b77-51fc-485d-854f-212b84ac731b None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:24:11.511393 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 245/494] 10.222.0.33 () {56 vars in 1109 bytes} [Mon Jul 25 05:24:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:24:11.512742 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e340c2-79b0-47db-b12a-5ea9afd8d555 None None] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:24:11.513140 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e340c2-79b0-47db-b12a-5ea9afd8d555 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:24:11.513424 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e340c2-79b0-47db-b12a-5ea9afd8d555 None None] PATH_INFO: `/` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:24:11.514813 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 246/495] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jul 25 05:24:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:24:11.515357 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e340c2-79b0-47db-b12a-5ea9afd8d555 None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:24:11.573090 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:24:11.577959 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-637a0a97-b6d6-4c32-9570-dc2f9e207baf None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:24:11.607508 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:24:11.616929 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30458aa6-1d48-4482-ad4a-b0a0aed3de0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:24:11.617600 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30458aa6-1d48-4482-ad4a-b0a0aed3de0c None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:24:11.617886 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30458aa6-1d48-4482-ad4a-b0a0aed3de0c None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:24:11.618164 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30458aa6-1d48-4482-ad4a-b0a0aed3de0c None admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:24:11.618910 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30458aa6-1d48-4482-ad4a-b0a0aed3de0c None admin] RBAC: Adding query filter params (name=service) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:24:11.619217 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30458aa6-1d48-4482-ad4a-b0a0aed3de0c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:24:11.620592 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30458aa6-1d48-4482-ad4a-b0a0aed3de0c None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:24:11.629390 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 250/496] 10.222.0.33 () {56 vars in 1103 bytes} [Mon Jul 25 05:24:11 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:24:11.630168 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30458aa6-1d48-4482-ad4a-b0a0aed3de0c None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:24:11.634972 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e340c2-79b0-47db-b12a-5ea9afd8d555 None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:24:11.671693 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac79f1e5-5636-4f7d-a480-87d26cd57ae8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:24:11.672356 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac79f1e5-5636-4f7d-a480-87d26cd57ae8 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:24:11.672945 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac79f1e5-5636-4f7d-a480-87d26cd57ae8 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:24:11.672945 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac79f1e5-5636-4f7d-a480-87d26cd57ae8 None admin] PATH_INFO: `/v3/domains` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:24:11.673672 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac79f1e5-5636-4f7d-a480-87d26cd57ae8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:24:11.673982 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac79f1e5-5636-4f7d-a480-87d26cd57ae8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:24:11.675355 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac79f1e5-5636-4f7d-a480-87d26cd57ae8 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:24:11.688297 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 247/497] 10.222.0.33 () {56 vars in 1109 bytes} [Mon Jul 25 05:24:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:24:11.689670 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5ace0d-dae1-497e-b7cc-3b01aa8b272e None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:24:11.689964 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5ace0d-dae1-497e-b7cc-3b01aa8b272e None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:24:11.690248 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5ace0d-dae1-497e-b7cc-3b01aa8b272e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:24:11.707449 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-dc5ace0d-dae1-497e-b7cc-3b01aa8b272e None None] MFA Rules not processed for user `1eae540242d94c10b0ad33960a855f08`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:24:11.709341 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-dc5ace0d-dae1-497e-b7cc-3b01aa8b272e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:24:11.741639 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30458aa6-1d48-4482-ad4a-b0a0aed3de0c None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:24:11.778236 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-529ad1ad-094c-4a16-96ec-b22299f93cba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:24:11.779427 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529ad1ad-094c-4a16-96ec-b22299f93cba None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:24:11.779913 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529ad1ad-094c-4a16-96ec-b22299f93cba None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:24:11.780430 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 248/498] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jul 25 05:24:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 05:24:11.780854 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529ad1ad-094c-4a16-96ec-b22299f93cba None admin] PATH_INFO: `/v3/users/glance` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:24:11.781202 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc5ace0d-dae1-497e-b7cc-3b01aa8b272e None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:24:11.797675 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-529ad1ad-094c-4a16-96ec-b22299f93cba None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:24:11.799877 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-529ad1ad-094c-4a16-96ec-b22299f93cba None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:24:11.821673 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application [None req-529ad1ad-094c-4a16-96ec-b22299f93cba None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jul 25 05:24:11.821673 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:24:11.821673 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:24:11.821673 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:24:11.821673 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:24:11.821673 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:24:11.821673 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:24:11.821673 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:24:11.821673 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:24:11.821673 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:24:11.821673 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:24:11.821673 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:24:11.821673 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 25 05:24:11.821673 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 25 05:24:11.821673 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 25 05:24:11.821673 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 25 05:24:11.822604 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:24:11.822604 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:24:11.822604 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 25 05:24:11.822604 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 05:24:11.822604 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 25 05:24:11.822604 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 05:24:11.822604 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:24:11.822604 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:24:11.822604 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:24:11.822604 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:24:11.822604 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:24:11.822604 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application with Lock( Jul 25 05:24:11.822604 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:24:11.822604 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:24:11.822604 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:24:11.822604 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:24:11.822604 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:24:11.822604 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:24:11.823543 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:24:11.823543 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:24:11.823543 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 25 05:24:11.823543 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 25 05:24:11.823543 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 25 05:24:11.823543 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 25 05:24:11.823543 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 25 05:24:11.823543 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 25 05:24:11.823543 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jul 25 05:24:11.823543 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Jul 25 05:24:11.823543 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 251/499] 10.222.0.33 () {56 vars in 1099 bytes} [Mon Jul 25 05:24:11 2022] GET /identity/v3/users/glance => generated 84 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:24:11.824111 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-529ad1ad-094c-4a16-96ec-b22299f93cba None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:24:11.832217 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:24:11.876622 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:24:11.898339 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc5ace0d-dae1-497e-b7cc-3b01aa8b272e None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:24:11.926530 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-529ad1ad-094c-4a16-96ec-b22299f93cba None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:24:11.931088 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce447827-e25c-4156-9431-4a09c4b4179a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:24:11.931880 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce447827-e25c-4156-9431-4a09c4b4179a None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:24:11.932163 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce447827-e25c-4156-9431-4a09c4b4179a None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:24:11.932439 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce447827-e25c-4156-9431-4a09c4b4179a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:24:11.943150 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce447827-e25c-4156-9431-4a09c4b4179a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:24:11.944590 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce447827-e25c-4156-9431-4a09c4b4179a None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:24:11.960934 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d159d12c-997d-4453-8910-051b2c388b93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:24:11.961718 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d159d12c-997d-4453-8910-051b2c388b93 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:24:11.962134 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d159d12c-997d-4453-8910-051b2c388b93 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:24:11.962417 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application [None req-ce447827-e25c-4156-9431-4a09c4b4179a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 05:24:11.962417 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:24:11.962417 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:24:11.962417 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:24:11.962417 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:24:11.962417 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:24:11.962417 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:24:11.962417 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:24:11.962417 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:24:11.962417 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:24:11.962417 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:24:11.962417 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:24:11.962417 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 05:24:11.962417 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 05:24:11.962417 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 05:24:11.963430 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 05:24:11.963430 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:24:11.963430 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:24:11.963430 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:24:11.963430 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:24:11.963430 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:24:11.963430 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:24:11.963430 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:24:11.963430 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application with Lock( Jul 25 05:24:11.963430 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:24:11.963430 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:24:11.963430 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:24:11.963430 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:24:11.963430 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:24:11.963430 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:24:11.963430 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:24:11.963430 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:24:11.964493 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 05:24:11.964493 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 05:24:11.964493 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 05:24:11.964493 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Jul 25 05:24:11.964749 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d159d12c-997d-4453-8910-051b2c388b93 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:24:11.964870 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 249/500] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Jul 25 05:24:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:24:11.964870 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce447827-e25c-4156-9431-4a09c4b4179a None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:24:11.972384 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d159d12c-997d-4453-8910-051b2c388b93 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:24:11.973873 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d159d12c-997d-4453-8910-051b2c388b93 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:24:11.987668 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application [None req-d159d12c-997d-4453-8910-051b2c388b93 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 25 05:24:11.987668 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:24:11.987668 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:24:11.987668 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:24:11.987668 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:24:11.987668 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:24:11.987668 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:24:11.987668 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:24:11.987668 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:24:11.987668 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:24:11.987668 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:24:11.987668 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:24:11.987668 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 25 05:24:11.987668 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 25 05:24:11.987668 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 25 05:24:11.987668 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 25 05:24:11.988726 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:24:11.988726 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:24:11.988726 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 25 05:24:11.988726 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 05:24:11.988726 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 25 05:24:11.988726 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 05:24:11.988726 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:24:11.988726 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:24:11.988726 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:24:11.988726 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:24:11.988726 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:24:11.988726 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application with Lock( Jul 25 05:24:11.988726 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:24:11.988726 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:24:11.988726 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:24:11.988726 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:24:11.988726 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:24:11.988726 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:24:11.994374 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:24:11.994374 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:24:11.994374 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 25 05:24:11.994374 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 25 05:24:11.994374 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 25 05:24:11.994374 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 25 05:24:11.994374 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 25 05:24:11.994374 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 25 05:24:11.994374 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 25 05:24:11.994374 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Jul 25 05:24:11.994374 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 252/501] 10.222.0.33 () {56 vars in 1093 bytes} [Mon Jul 25 05:24:11 2022] GET /identity/v3/users/nova => generated 82 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:24:11.994374 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d159d12c-997d-4453-8910-051b2c388b93 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:24:12.056476 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:24:12.067034 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce447827-e25c-4156-9431-4a09c4b4179a None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:24:12.089234 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d159d12c-997d-4453-8910-051b2c388b93 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:24:12.099750 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:24:12.101501 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d49c4e7b-493d-481f-b54d-e5f012bc2fec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:24:12.102294 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49c4e7b-493d-481f-b54d-e5f012bc2fec None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:24:12.102578 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49c4e7b-493d-481f-b54d-e5f012bc2fec None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:24:12.102859 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49c4e7b-493d-481f-b54d-e5f012bc2fec None admin] PATH_INFO: `/v3/roles/service` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:24:12.113051 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d49c4e7b-493d-481f-b54d-e5f012bc2fec None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 05:24:12.120574 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application [None req-d49c4e7b-493d-481f-b54d-e5f012bc2fec None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 25 05:24:12.120574 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:24:12.120574 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:24:12.120574 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:24:12.120574 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:24:12.120574 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:24:12.120574 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:24:12.120574 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:24:12.120574 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:24:12.120574 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:24:12.120574 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:24:12.120574 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:24:12.120574 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 05:24:12.120574 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 05:24:12.120574 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 05:24:12.120574 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 05:24:12.121718 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 05:24:12.121718 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 05:24:12.121718 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 05:24:12.121718 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 05:24:12.121718 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:24:12.121718 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:24:12.121718 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:24:12.121718 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:24:12.121718 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:24:12.121718 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:24:12.121718 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:24:12.121718 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application with Lock( Jul 25 05:24:12.121718 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:24:12.121718 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:24:12.121718 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:24:12.121718 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:24:12.122695 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:24:12.122695 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:24:12.122695 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:24:12.122695 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:24:12.122695 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 05:24:12.122695 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 05:24:12.122695 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 05:24:12.122695 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 05:24:12.122695 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 05:24:12.122695 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 05:24:12.122695 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 25 05:24:12.122695 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Jul 25 05:24:12.122695 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 250/502] 10.222.0.33 () {56 vars in 1102 bytes} [Mon Jul 25 05:24:11 2022] GET /identity/v3/roles/service => generated 85 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:24:12.123564 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c19854-8c7b-44f4-a220-b439c8ed309c None None] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:24:12.123654 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c19854-8c7b-44f4-a220-b439c8ed309c None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:24:12.124038 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c19854-8c7b-44f4-a220-b439c8ed309c None None] PATH_INFO: `/` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:24:12.125294 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 251/503] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jul 25 05:24:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:24:12.127911 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ceadb5-7c43-4543-80f6-2c6a088b495e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:24:12.128613 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ceadb5-7c43-4543-80f6-2c6a088b495e None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:24:12.128823 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c19854-8c7b-44f4-a220-b439c8ed309c None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:24:12.128928 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ceadb5-7c43-4543-80f6-2c6a088b495e None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:24:12.129186 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ceadb5-7c43-4543-80f6-2c6a088b495e None admin] PATH_INFO: `/v3/users` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:24:12.130029 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84ceadb5-7c43-4543-80f6-2c6a088b495e None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:24:12.130336 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84ceadb5-7c43-4543-80f6-2c6a088b495e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:24:12.131769 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84ceadb5-7c43-4543-80f6-2c6a088b495e None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:24:12.168134 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 253/504] 10.222.0.33 () {56 vars in 1137 bytes} [Mon Jul 25 05:24:11 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:24:12.169026 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ceadb5-7c43-4543-80f6-2c6a088b495e None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:24:12.230742 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c19854-8c7b-44f4-a220-b439c8ed309c None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:24:12.262998 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f26979-bbed-431f-af59-5bef36440cb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:24:12.263708 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f26979-bbed-431f-af59-5bef36440cb8 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:24:12.263996 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f26979-bbed-431f-af59-5bef36440cb8 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:24:12.264269 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f26979-bbed-431f-af59-5bef36440cb8 None admin] PATH_INFO: `/v3/domains` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:24:12.265009 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25f26979-bbed-431f-af59-5bef36440cb8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:24:12.265315 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25f26979-bbed-431f-af59-5bef36440cb8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:24:12.266686 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25f26979-bbed-431f-af59-5bef36440cb8 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:24:12.267120 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ceadb5-7c43-4543-80f6-2c6a088b495e None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:24:12.278890 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 252/505] 10.222.0.33 () {56 vars in 1109 bytes} [Mon Jul 25 05:24:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:24:12.279687 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f26979-bbed-431f-af59-5bef36440cb8 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:24:12.292323 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:24:12.301922 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a21ef142-d942-4e24-b676-c79d98a16dfc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:24:12.302587 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21ef142-d942-4e24-b676-c79d98a16dfc None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:24:12.302874 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21ef142-d942-4e24-b676-c79d98a16dfc None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:24:12.303151 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21ef142-d942-4e24-b676-c79d98a16dfc None admin] PATH_INFO: `/v3/users` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:24:12.303985 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a21ef142-d942-4e24-b676-c79d98a16dfc None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:24:12.304297 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a21ef142-d942-4e24-b676-c79d98a16dfc None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:24:12.305760 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a21ef142-d942-4e24-b676-c79d98a16dfc None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:24:12.323623 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 254/506] 10.222.0.33 () {56 vars in 1133 bytes} [Mon Jul 25 05:24:12 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:24:12.324892 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44228436-8d28-4e47-96d3-f35a878f7188 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:24:12.325180 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44228436-8d28-4e47-96d3-f35a878f7188 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:24:12.325627 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44228436-8d28-4e47-96d3-f35a878f7188 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:24:12.341313 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-44228436-8d28-4e47-96d3-f35a878f7188 None None] MFA Rules not processed for user `1eae540242d94c10b0ad33960a855f08`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:24:12.342497 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-44228436-8d28-4e47-96d3-f35a878f7188 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:24:12.362971 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:24:12.384148 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f26979-bbed-431f-af59-5bef36440cb8 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:24:12.409814 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 255/507] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jul 25 05:24:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 05:24:12.410972 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44228436-8d28-4e47-96d3-f35a878f7188 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:24:12.417379 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f7dc14-df89-412e-84a4-ab84126fe3dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:24:12.418045 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f7dc14-df89-412e-84a4-ab84126fe3dc None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:24:12.418854 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f7dc14-df89-412e-84a4-ab84126fe3dc None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:24:12.419530 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f7dc14-df89-412e-84a4-ab84126fe3dc None admin] PATH_INFO: `/v3/roles` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:24:12.420286 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30f7dc14-df89-412e-84a4-ab84126fe3dc None admin] RBAC: Adding query filter params (name=service) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:24:12.420590 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30f7dc14-df89-412e-84a4-ab84126fe3dc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:24:12.423028 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30f7dc14-df89-412e-84a4-ab84126fe3dc None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:24:12.435342 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 253/508] 10.222.0.33 () {56 vars in 1103 bytes} [Mon Jul 25 05:24:12 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:24:12.435342 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f7dc14-df89-412e-84a4-ab84126fe3dc None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:24:12.505219 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:24:12.513117 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44228436-8d28-4e47-96d3-f35a878f7188 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:24:12.536983 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f7dc14-df89-412e-84a4-ab84126fe3dc None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:24:12.547995 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe5fd58-93ca-465c-a0e1-5ccfdb9d371e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:24:12.548655 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe5fd58-93ca-465c-a0e1-5ccfdb9d371e None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:24:12.548979 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe5fd58-93ca-465c-a0e1-5ccfdb9d371e None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:24:12.549286 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe5fd58-93ca-465c-a0e1-5ccfdb9d371e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:24:12.570981 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b2bb8c-4671-4ba7-a84e-028d1099b9de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:24:12.571766 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b2bb8c-4671-4ba7-a84e-028d1099b9de None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:24:12.572045 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b2bb8c-4671-4ba7-a84e-028d1099b9de None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:24:12.572325 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b2bb8c-4671-4ba7-a84e-028d1099b9de None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:24:12.574759 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:24:12.579974 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1b2bb8c-4671-4ba7-a84e-028d1099b9de None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:24:12.581448 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1b2bb8c-4671-4ba7-a84e-028d1099b9de None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:24:12.596982 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application [None req-e1b2bb8c-4671-4ba7-a84e-028d1099b9de None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 25 05:24:12.596982 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:24:12.596982 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:24:12.596982 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:24:12.596982 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:24:12.596982 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:24:12.596982 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:24:12.596982 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:24:12.596982 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:24:12.596982 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:24:12.596982 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:24:12.596982 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:24:12.596982 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 25 05:24:12.596982 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 25 05:24:12.596982 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 25 05:24:12.596982 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 25 05:24:12.598066 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:24:12.598066 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:24:12.598066 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 25 05:24:12.598066 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 05:24:12.598066 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 25 05:24:12.598066 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 05:24:12.598066 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:24:12.598066 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:24:12.598066 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:24:12.598066 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:24:12.598066 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:24:12.598066 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application with Lock( Jul 25 05:24:12.598066 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:24:12.598066 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:24:12.598066 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:24:12.598066 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:24:12.598066 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:24:12.598066 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:24:12.599260 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:24:12.599260 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:24:12.599260 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 25 05:24:12.599260 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 25 05:24:12.599260 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 25 05:24:12.599260 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 25 05:24:12.599260 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 25 05:24:12.599260 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 25 05:24:12.599260 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 25 05:24:12.599260 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Jul 25 05:24:12.599260 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 254/509] 10.222.0.33 () {56 vars in 1099 bytes} [Mon Jul 25 05:24:12 2022] GET /identity/v3/users/cinder => generated 84 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:24:12.599260 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b2bb8c-4671-4ba7-a84e-028d1099b9de None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:24:12.620821 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efe5fd58-93ca-465c-a0e1-5ccfdb9d371e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:24:12.622834 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efe5fd58-93ca-465c-a0e1-5ccfdb9d371e None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:24:12.703054 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b2bb8c-4671-4ba7-a84e-028d1099b9de None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:24:12.720871 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 256/510] 10.222.0.33 () {58 vars in 1282 bytes} [Mon Jul 25 05:24:12 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 25 05:24:12.721706 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe5fd58-93ca-465c-a0e1-5ccfdb9d371e None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:24:12.735477 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:24:12.741421 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8641fb92-907e-40b4-81c4-c65ce243fa27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:24:12.742069 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8641fb92-907e-40b4-81c4-c65ce243fa27 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:24:12.742352 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8641fb92-907e-40b4-81c4-c65ce243fa27 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:24:12.742625 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8641fb92-907e-40b4-81c4-c65ce243fa27 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:24:12.776655 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:24:12.812059 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8641fb92-907e-40b4-81c4-c65ce243fa27 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:24:12.813563 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8641fb92-907e-40b4-81c4-c65ce243fa27 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:24:12.820404 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe5fd58-93ca-465c-a0e1-5ccfdb9d371e None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:24:12.853941 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7941465a-2c4d-4df2-b3bf-07326bcfccc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:24:12.854669 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7941465a-2c4d-4df2-b3bf-07326bcfccc8 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:24:12.854963 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7941465a-2c4d-4df2-b3bf-07326bcfccc8 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:24:12.855240 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7941465a-2c4d-4df2-b3bf-07326bcfccc8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:24:12.865237 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7941465a-2c4d-4df2-b3bf-07326bcfccc8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:24:12.866671 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7941465a-2c4d-4df2-b3bf-07326bcfccc8 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:24:12.884039 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application [None req-7941465a-2c4d-4df2-b3bf-07326bcfccc8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 05:24:12.884039 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:24:12.884039 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:24:12.884039 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:24:12.884039 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:24:12.884039 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:24:12.884039 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:24:12.884039 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:24:12.884039 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:24:12.884039 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:24:12.884039 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:24:12.884039 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:24:12.884039 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 05:24:12.884039 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 05:24:12.884039 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 05:24:12.884935 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 05:24:12.884935 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:24:12.884935 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:24:12.884935 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:24:12.884935 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:24:12.884935 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:24:12.884935 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:24:12.884935 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:24:12.884935 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application with Lock( Jul 25 05:24:12.884935 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:24:12.884935 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:24:12.884935 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:24:12.884935 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:24:12.884935 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:24:12.884935 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:24:12.884935 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:24:12.884935 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:24:12.885812 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 05:24:12.885812 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 05:24:12.885812 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 05:24:12.885812 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Jul 25 05:24:12.885812 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 257/511] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Jul 25 05:24:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:24:12.886088 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7941465a-2c4d-4df2-b3bf-07326bcfccc8 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:24:12.911717 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 255/512] 10.222.0.33 () {58 vars in 1282 bytes} [Mon Jul 25 05:24:12 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 25 05:24:12.912558 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8641fb92-907e-40b4-81c4-c65ce243fa27 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:24:12.942760 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:24:12.991738 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7941465a-2c4d-4df2-b3bf-07326bcfccc8 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:24:13.000522 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:24:13.014169 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8641fb92-907e-40b4-81c4-c65ce243fa27 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:24:13.025253 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d8b4cd7-af35-44e4-9115-5214e366db14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:24:13.026044 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8b4cd7-af35-44e4-9115-5214e366db14 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:24:13.026333 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8b4cd7-af35-44e4-9115-5214e366db14 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:24:13.026617 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8b4cd7-af35-44e4-9115-5214e366db14 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:24:13.037980 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d8b4cd7-af35-44e4-9115-5214e366db14 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:24:13.039503 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d8b4cd7-af35-44e4-9115-5214e366db14 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:24:13.048218 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a77dc0a7-6a31-4cdb-9d86-2602981c4439 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:24:13.048899 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77dc0a7-6a31-4cdb-9d86-2602981c4439 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:24:13.049183 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77dc0a7-6a31-4cdb-9d86-2602981c4439 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:24:13.049457 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77dc0a7-6a31-4cdb-9d86-2602981c4439 None admin] PATH_INFO: `/v3/users` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:24:13.050288 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a77dc0a7-6a31-4cdb-9d86-2602981c4439 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:24:13.050595 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a77dc0a7-6a31-4cdb-9d86-2602981c4439 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:24:13.052013 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a77dc0a7-6a31-4cdb-9d86-2602981c4439 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:24:13.056360 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application [None req-8d8b4cd7-af35-44e4-9115-5214e366db14 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 05:24:13.056360 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:24:13.056360 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:24:13.056360 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:24:13.056360 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:24:13.056360 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:24:13.056360 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:24:13.056360 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:24:13.056360 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:24:13.056360 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:24:13.056360 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:24:13.056360 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:24:13.056360 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 05:24:13.056360 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 05:24:13.056360 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 05:24:13.057216 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 05:24:13.057216 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:24:13.057216 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:24:13.057216 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:24:13.057216 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:24:13.057216 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:24:13.057216 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:24:13.057216 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:24:13.057216 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application with Lock( Jul 25 05:24:13.057216 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:24:13.057216 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:24:13.057216 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:24:13.057216 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:24:13.057216 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:24:13.057216 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:24:13.057216 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:24:13.057216 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:24:13.058112 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 05:24:13.058112 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 05:24:13.058112 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 05:24:13.058112 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Jul 25 05:24:13.058112 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 258/513] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Jul 25 05:24:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:24:13.058792 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d8b4cd7-af35-44e4-9115-5214e366db14 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:24:13.088638 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 256/514] 10.222.0.33 () {56 vars in 1137 bytes} [Mon Jul 25 05:24:12 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:24:13.089515 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a77dc0a7-6a31-4cdb-9d86-2602981c4439 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:24:13.159989 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d8b4cd7-af35-44e4-9115-5214e366db14 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:24:13.180596 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:24:13.191497 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a77dc0a7-6a31-4cdb-9d86-2602981c4439 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:24:13.194983 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-399a93dc-6383-4ae6-bd1b-2dc354431bd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:24:13.195881 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399a93dc-6383-4ae6-bd1b-2dc354431bd5 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:24:13.196176 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399a93dc-6383-4ae6-bd1b-2dc354431bd5 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:24:13.196458 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399a93dc-6383-4ae6-bd1b-2dc354431bd5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:24:13.206411 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-399a93dc-6383-4ae6-bd1b-2dc354431bd5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:24:13.207860 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-399a93dc-6383-4ae6-bd1b-2dc354431bd5 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:24:13.224101 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-197b7ae1-9293-43d4-9bc3-5c041547cae5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:24:13.224753 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197b7ae1-9293-43d4-9bc3-5c041547cae5 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:24:13.225063 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197b7ae1-9293-43d4-9bc3-5c041547cae5 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:24:13.225339 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197b7ae1-9293-43d4-9bc3-5c041547cae5 None admin] PATH_INFO: `/v3/domains` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:24:13.225430 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application [None req-399a93dc-6383-4ae6-bd1b-2dc354431bd5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 05:24:13.225430 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:24:13.225430 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:24:13.225430 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:24:13.225430 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:24:13.225430 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:24:13.225430 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:24:13.225430 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:24:13.225430 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:24:13.225430 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:24:13.225430 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:24:13.225430 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:24:13.225430 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 05:24:13.225430 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 05:24:13.225430 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 05:24:13.226357 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 05:24:13.226357 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:24:13.226357 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:24:13.226357 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:24:13.226357 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:24:13.226357 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:24:13.226357 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:24:13.226357 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:24:13.226357 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application with Lock( Jul 25 05:24:13.226357 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:24:13.226357 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:24:13.226357 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:24:13.226357 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:24:13.226357 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:24:13.226357 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:24:13.226357 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:24:13.226357 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:24:13.227261 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 05:24:13.227261 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 05:24:13.227261 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 05:24:13.227261 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Jul 25 05:24:13.227485 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-197b7ae1-9293-43d4-9bc3-5c041547cae5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:24:13.227485 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-197b7ae1-9293-43d4-9bc3-5c041547cae5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:24:13.227616 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 259/515] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Jul 25 05:24:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:24:13.227682 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-399a93dc-6383-4ae6-bd1b-2dc354431bd5 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:24:13.227848 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-197b7ae1-9293-43d4-9bc3-5c041547cae5 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:24:13.242051 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 257/516] 10.222.0.33 () {56 vars in 1109 bytes} [Mon Jul 25 05:24:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:24:13.242771 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-197b7ae1-9293-43d4-9bc3-5c041547cae5 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:24:13.246895 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:24:13.329992 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-399a93dc-6383-4ae6-bd1b-2dc354431bd5 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:24:13.339870 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-197b7ae1-9293-43d4-9bc3-5c041547cae5 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:24:13.363346 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a84c60e-d9ea-497c-ace6-0ad4fb1fe2f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:24:13.364140 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a84c60e-d9ea-497c-ace6-0ad4fb1fe2f7 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:24:13.364429 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a84c60e-d9ea-497c-ace6-0ad4fb1fe2f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:24:13.364709 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a84c60e-d9ea-497c-ace6-0ad4fb1fe2f7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:24:13.374101 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-571b14cf-806f-43cb-8cf9-2f2f7a5bac6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:24:13.374561 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a84c60e-d9ea-497c-ace6-0ad4fb1fe2f7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:24:13.374754 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571b14cf-806f-43cb-8cf9-2f2f7a5bac6a None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:24:13.375040 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571b14cf-806f-43cb-8cf9-2f2f7a5bac6a None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:24:13.375325 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571b14cf-806f-43cb-8cf9-2f2f7a5bac6a None admin] PATH_INFO: `/v3/domains` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:24:13.376005 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a84c60e-d9ea-497c-ace6-0ad4fb1fe2f7 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:24:13.376101 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-571b14cf-806f-43cb-8cf9-2f2f7a5bac6a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:24:13.376329 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-571b14cf-806f-43cb-8cf9-2f2f7a5bac6a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:24:13.377710 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-571b14cf-806f-43cb-8cf9-2f2f7a5bac6a None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:24:13.389243 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 258/517] 10.222.0.33 () {56 vars in 1109 bytes} [Mon Jul 25 05:24:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:24:13.390022 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-571b14cf-806f-43cb-8cf9-2f2f7a5bac6a None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:24:13.394057 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application [None req-1a84c60e-d9ea-497c-ace6-0ad4fb1fe2f7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 05:24:13.394057 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:24:13.394057 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:24:13.394057 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:24:13.394057 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:24:13.394057 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:24:13.394057 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:24:13.394057 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:24:13.394057 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:24:13.394057 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:24:13.394057 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:24:13.394057 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:24:13.394057 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 05:24:13.394057 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 05:24:13.394057 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 05:24:13.395061 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 05:24:13.395061 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:24:13.395061 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:24:13.395061 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:24:13.395061 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:24:13.395061 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:24:13.395061 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:24:13.395061 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:24:13.395061 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application with Lock( Jul 25 05:24:13.395061 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:24:13.395061 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:24:13.395061 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:24:13.395061 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:24:13.395061 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:24:13.395061 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:24:13.395061 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:24:13.395061 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:24:13.396019 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 05:24:13.396019 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 05:24:13.396019 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 05:24:13.396019 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Jul 25 05:24:13.396019 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 260/518] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Jul 25 05:24:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:24:13.396339 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a84c60e-d9ea-497c-ace6-0ad4fb1fe2f7 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:24:13.401625 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:24:13.458389 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:24:13.492018 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-571b14cf-806f-43cb-8cf9-2f2f7a5bac6a None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:24:13.495840 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a84c60e-d9ea-497c-ace6-0ad4fb1fe2f7 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:24:13.527273 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a24a8cb1-efdb-4b76-ab6d-868aef4897b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:24:13.528418 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24a8cb1-efdb-4b76-ab6d-868aef4897b5 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:24:13.528714 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24a8cb1-efdb-4b76-ab6d-868aef4897b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:24:13.529023 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24a8cb1-efdb-4b76-ab6d-868aef4897b5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:24:13.529512 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f12218e-01fd-4129-8f6f-394e0420ab54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:24:13.530171 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f12218e-01fd-4129-8f6f-394e0420ab54 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:24:13.530454 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f12218e-01fd-4129-8f6f-394e0420ab54 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:24:13.530728 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f12218e-01fd-4129-8f6f-394e0420ab54 None admin] PATH_INFO: `/v3/domains` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:24:13.531459 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f12218e-01fd-4129-8f6f-394e0420ab54 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:24:13.531790 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f12218e-01fd-4129-8f6f-394e0420ab54 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:24:13.533152 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f12218e-01fd-4129-8f6f-394e0420ab54 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:24:13.546135 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 261/519] 10.222.0.33 () {56 vars in 1109 bytes} [Mon Jul 25 05:24:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:24:13.546939 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f12218e-01fd-4129-8f6f-394e0420ab54 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:24:13.602279 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a24a8cb1-efdb-4b76-ab6d-868aef4897b5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:24:13.604101 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a24a8cb1-efdb-4b76-ab6d-868aef4897b5 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:24:13.616172 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:24:13.643907 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f12218e-01fd-4129-8f6f-394e0420ab54 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:24:13.666720 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:24:13.677935 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f1c3c3-9212-4e73-8f5d-f5f7691fac4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:24:13.678603 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f1c3c3-9212-4e73-8f5d-f5f7691fac4a None admin] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:24:13.678886 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f1c3c3-9212-4e73-8f5d-f5f7691fac4a None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:24:13.679162 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f1c3c3-9212-4e73-8f5d-f5f7691fac4a None admin] PATH_INFO: `/v3/users` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:24:13.679864 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66f1c3c3-9212-4e73-8f5d-f5f7691fac4a None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:24:13.681335 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66f1c3c3-9212-4e73-8f5d-f5f7691fac4a None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:24:13.706871 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 262/520] 10.222.0.33 () {60 vars in 1132 bytes} [Mon Jul 25 05:24:13 2022] POST /identity/v3/users => generated 247 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 05:24:13.706871 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f1c3c3-9212-4e73-8f5d-f5f7691fac4a None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:24:13.712913 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 259/521] 10.222.0.33 () {58 vars in 1282 bytes} [Mon Jul 25 05:24:13 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 25 05:24:13.713668 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a24a8cb1-efdb-4b76-ab6d-868aef4897b5 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:24:13.831343 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a24a8cb1-efdb-4b76-ab6d-868aef4897b5 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:24:13.837733 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f1c3c3-9212-4e73-8f5d-f5f7691fac4a None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:24:13.843898 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:24:13.873886 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4cc2663-7de2-4059-bba4-6f011ab543e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:24:13.874555 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4cc2663-7de2-4059-bba4-6f011ab543e4 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:24:13.874845 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4cc2663-7de2-4059-bba4-6f011ab543e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:24:13.875129 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4cc2663-7de2-4059-bba4-6f011ab543e4 None admin] PATH_INFO: `/v3/domains` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:24:13.875864 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4cc2663-7de2-4059-bba4-6f011ab543e4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:24:13.876176 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4cc2663-7de2-4059-bba4-6f011ab543e4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:24:13.877089 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b0fff1-4806-4663-ade8-f5b5d9840ea7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:24:13.877606 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4cc2663-7de2-4059-bba4-6f011ab543e4 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:24:13.877890 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b0fff1-4806-4663-ade8-f5b5d9840ea7 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:24:13.878175 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b0fff1-4806-4663-ade8-f5b5d9840ea7 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:24:13.878453 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b0fff1-4806-4663-ade8-f5b5d9840ea7 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:24:13.888343 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9b0fff1-4806-4663-ade8-f5b5d9840ea7 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:24:13.890224 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9b0fff1-4806-4663-ade8-f5b5d9840ea7 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:24:13.894551 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 260/522] 10.222.0.33 () {56 vars in 1109 bytes} [Mon Jul 25 05:24:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:24:13.895252 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4cc2663-7de2-4059-bba4-6f011ab543e4 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:24:13.910116 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application [None req-a9b0fff1-4806-4663-ade8-f5b5d9840ea7 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jul 25 05:24:13.910116 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:24:13.910116 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:24:13.910116 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:24:13.910116 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:24:13.910116 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:24:13.910116 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:24:13.910116 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:24:13.910116 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:24:13.910116 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:24:13.910116 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:24:13.910116 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:24:13.910116 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 25 05:24:13.910116 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 25 05:24:13.910116 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 25 05:24:13.910116 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 25 05:24:13.911058 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:24:13.911058 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:24:13.911058 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 25 05:24:13.911058 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 05:24:13.911058 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 25 05:24:13.911058 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 05:24:13.911058 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:24:13.911058 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:24:13.911058 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:24:13.911058 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:24:13.911058 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:24:13.911058 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application with Lock( Jul 25 05:24:13.911058 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:24:13.911058 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:24:13.911058 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:24:13.911058 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:24:13.911058 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:24:13.911058 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:24:13.912409 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:24:13.912409 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:24:13.912409 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 25 05:24:13.912409 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 25 05:24:13.912409 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 25 05:24:13.912409 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 25 05:24:13.912409 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 25 05:24:13.912409 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 25 05:24:13.912409 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jul 25 05:24:13.912409 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Jul 25 05:24:13.912409 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 263/523] 10.222.0.33 () {56 vars in 1102 bytes} [Mon Jul 25 05:24:13 2022] GET /identity/v3/users/neutron => generated 85 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:24:13.912409 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b0fff1-4806-4663-ade8-f5b5d9840ea7 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:24:13.962403 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:24:14.005014 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4cc2663-7de2-4059-bba4-6f011ab543e4 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:24:14.030377 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b0fff1-4806-4663-ade8-f5b5d9840ea7 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:24:14.039733 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-825c8b30-add7-4199-8ccb-8377469ca953 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:24:14.040534 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825c8b30-add7-4199-8ccb-8377469ca953 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:24:14.040849 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825c8b30-add7-4199-8ccb-8377469ca953 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:24:14.041137 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825c8b30-add7-4199-8ccb-8377469ca953 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:24:14.048716 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-825c8b30-add7-4199-8ccb-8377469ca953 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:24:14.050191 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-825c8b30-add7-4199-8ccb-8377469ca953 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:24:14.066143 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application [None req-825c8b30-add7-4199-8ccb-8377469ca953 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 05:24:14.066143 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:24:14.066143 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:24:14.066143 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:24:14.066143 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:24:14.066143 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:24:14.066143 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:24:14.066143 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:24:14.066143 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:24:14.066143 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:24:14.066143 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:24:14.066143 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:24:14.066143 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 25 05:24:14.066143 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 25 05:24:14.066143 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 25 05:24:14.067010 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 25 05:24:14.067010 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:24:14.067010 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:24:14.067010 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:24:14.067010 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:24:14.067010 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:24:14.067010 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:24:14.067010 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:24:14.067010 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application with Lock( Jul 25 05:24:14.067010 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:24:14.067010 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:24:14.067010 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:24:14.067010 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:24:14.067010 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:24:14.067010 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:24:14.067010 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:24:14.067010 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:24:14.073152 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 25 05:24:14.073152 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 25 05:24:14.073152 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 25 05:24:14.073152 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 25 05:24:14.073152 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 25 05:24:14.073152 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 25 05:24:14.073152 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 05:24:14.073152 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Jul 25 05:24:14.073587 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c76745e-0fea-43c5-a706-712b66122ffb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:24:14.073587 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c76745e-0fea-43c5-a706-712b66122ffb None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:24:14.073587 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c76745e-0fea-43c5-a706-712b66122ffb None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:24:14.073587 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c76745e-0fea-43c5-a706-712b66122ffb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:24:14.073839 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 261/524] 10.222.0.33 () {56 vars in 1111 bytes} [Mon Jul 25 05:24:13 2022] GET /identity/v3/projects/service => generated 88 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:24:14.073839 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-825c8b30-add7-4199-8ccb-8377469ca953 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:24:14.085182 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c76745e-0fea-43c5-a706-712b66122ffb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:24:14.086737 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c76745e-0fea-43c5-a706-712b66122ffb None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:24:14.103272 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:24:14.105264 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application [None req-8c76745e-0fea-43c5-a706-712b66122ffb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 05:24:14.105264 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:24:14.105264 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:24:14.105264 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:24:14.105264 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:24:14.105264 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:24:14.105264 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:24:14.105264 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:24:14.105264 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:24:14.105264 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:24:14.105264 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:24:14.105264 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:24:14.105264 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 05:24:14.105264 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 05:24:14.105264 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 05:24:14.106430 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 05:24:14.106430 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:24:14.106430 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:24:14.106430 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:24:14.106430 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:24:14.106430 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:24:14.106430 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:24:14.106430 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:24:14.106430 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application with Lock( Jul 25 05:24:14.106430 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:24:14.106430 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:24:14.106430 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:24:14.106430 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:24:14.106430 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:24:14.106430 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:24:14.106430 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:24:14.106430 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:24:14.107319 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 05:24:14.107319 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 05:24:14.107319 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 05:24:14.107319 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Jul 25 05:24:14.107319 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 264/525] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Jul 25 05:24:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:24:14.107675 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c76745e-0fea-43c5-a706-712b66122ffb None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:24:14.184896 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-825c8b30-add7-4199-8ccb-8377469ca953 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:24:14.205451 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:24:14.215620 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c76745e-0fea-43c5-a706-712b66122ffb None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:24:14.223460 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbfae8f1-7057-4934-9e1d-6683f2345c95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:24:14.224392 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbfae8f1-7057-4934-9e1d-6683f2345c95 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:24:14.224677 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbfae8f1-7057-4934-9e1d-6683f2345c95 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:24:14.225172 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbfae8f1-7057-4934-9e1d-6683f2345c95 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:24:14.231871 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbfae8f1-7057-4934-9e1d-6683f2345c95 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:24:14.233356 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbfae8f1-7057-4934-9e1d-6683f2345c95 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:24:14.250358 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee2d3fd-4b4a-4df4-a1c2-193302b9495d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:24:14.250686 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application [None req-cbfae8f1-7057-4934-9e1d-6683f2345c95 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 05:24:14.250686 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:24:14.250686 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:24:14.250686 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:24:14.250686 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:24:14.250686 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:24:14.250686 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:24:14.250686 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:24:14.250686 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:24:14.250686 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:24:14.250686 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:24:14.250686 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:24:14.250686 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 25 05:24:14.250686 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 25 05:24:14.250686 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 25 05:24:14.251514 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 25 05:24:14.251514 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:24:14.251514 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:24:14.251514 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:24:14.251514 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:24:14.251514 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:24:14.251514 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:24:14.251514 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:24:14.251514 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application with Lock( Jul 25 05:24:14.251514 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:24:14.251514 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:24:14.251514 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:24:14.251514 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:24:14.251514 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:24:14.251514 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:24:14.251514 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:24:14.251514 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:24:14.257515 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 25 05:24:14.257515 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 25 05:24:14.257515 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 25 05:24:14.257515 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 25 05:24:14.257515 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 25 05:24:14.257515 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 25 05:24:14.257515 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 05:24:14.257515 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Jul 25 05:24:14.257939 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee2d3fd-4b4a-4df4-a1c2-193302b9495d None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:24:14.257939 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee2d3fd-4b4a-4df4-a1c2-193302b9495d None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:24:14.257939 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee2d3fd-4b4a-4df4-a1c2-193302b9495d None admin] PATH_INFO: `/v3/users` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:24:14.258132 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 262/526] 10.222.0.33 () {56 vars in 1111 bytes} [Mon Jul 25 05:24:14 2022] GET /identity/v3/projects/service => generated 88 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:24:14.258201 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eee2d3fd-4b4a-4df4-a1c2-193302b9495d None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:24:14.258201 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eee2d3fd-4b4a-4df4-a1c2-193302b9495d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:24:14.258405 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbfae8f1-7057-4934-9e1d-6683f2345c95 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:24:14.258479 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eee2d3fd-4b4a-4df4-a1c2-193302b9495d None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:24:14.278704 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 265/527] 10.222.0.33 () {56 vars in 1139 bytes} [Mon Jul 25 05:24:14 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:24:14.278704 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee2d3fd-4b4a-4df4-a1c2-193302b9495d None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:24:14.360727 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:24:14.371686 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbfae8f1-7057-4934-9e1d-6683f2345c95 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:24:14.378416 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee2d3fd-4b4a-4df4-a1c2-193302b9495d None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:24:14.413575 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f665d5d-0f43-4fbe-bccf-bda2a9052700 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:24:14.414053 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc8bf6c-302b-4f17-b993-55f8fca27d5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:24:14.414250 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f665d5d-0f43-4fbe-bccf-bda2a9052700 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:24:14.414531 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f665d5d-0f43-4fbe-bccf-bda2a9052700 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:24:14.414725 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc8bf6c-302b-4f17-b993-55f8fca27d5d None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:24:14.414822 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f665d5d-0f43-4fbe-bccf-bda2a9052700 None admin] PATH_INFO: `/v3/domains` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:24:14.415002 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc8bf6c-302b-4f17-b993-55f8fca27d5d None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:24:14.415260 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc8bf6c-302b-4f17-b993-55f8fca27d5d None admin] PATH_INFO: `/v3/projects` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:24:14.415527 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f665d5d-0f43-4fbe-bccf-bda2a9052700 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:24:14.415840 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f665d5d-0f43-4fbe-bccf-bda2a9052700 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:24:14.416051 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fc8bf6c-302b-4f17-b993-55f8fca27d5d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:24:14.416359 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fc8bf6c-302b-4f17-b993-55f8fca27d5d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:24:14.417253 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f665d5d-0f43-4fbe-bccf-bda2a9052700 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:24:14.417908 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fc8bf6c-302b-4f17-b993-55f8fca27d5d None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:24:14.430451 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 266/528] 10.222.0.33 () {56 vars in 1109 bytes} [Mon Jul 25 05:24:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:24:14.431602 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f665d5d-0f43-4fbe-bccf-bda2a9052700 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:24:14.436820 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 263/529] 10.222.0.33 () {56 vars in 1148 bytes} [Mon Jul 25 05:24:14 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:24:14.437579 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc8bf6c-302b-4f17-b993-55f8fca27d5d None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:24:14.453750 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:24:14.542495 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc8bf6c-302b-4f17-b993-55f8fca27d5d None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:24:14.548169 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f665d5d-0f43-4fbe-bccf-bda2a9052700 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:24:14.581456 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-507ddbd5-84c0-47d5-9c9c-b832f8733191 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:24:14.582104 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507ddbd5-84c0-47d5-9c9c-b832f8733191 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:24:14.582391 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507ddbd5-84c0-47d5-9c9c-b832f8733191 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:24:14.582670 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507ddbd5-84c0-47d5-9c9c-b832f8733191 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:24:14.585207 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e662f5e-8b4d-4ae4-957b-c75111a9b4da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:24:14.585867 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e662f5e-8b4d-4ae4-957b-c75111a9b4da None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:24:14.586159 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e662f5e-8b4d-4ae4-957b-c75111a9b4da None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:24:14.586440 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e662f5e-8b4d-4ae4-957b-c75111a9b4da None admin] PATH_INFO: `/v3/projects` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:24:14.587227 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e662f5e-8b4d-4ae4-957b-c75111a9b4da None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:24:14.587537 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e662f5e-8b4d-4ae4-957b-c75111a9b4da None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:24:14.588992 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e662f5e-8b4d-4ae4-957b-c75111a9b4da None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:24:14.602968 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:24:14.607932 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 267/530] 10.222.0.33 () {56 vars in 1148 bytes} [Mon Jul 25 05:24:14 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:24:14.608762 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e662f5e-8b4d-4ae4-957b-c75111a9b4da None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:24:14.659683 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-507ddbd5-84c0-47d5-9c9c-b832f8733191 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:24:14.661174 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-507ddbd5-84c0-47d5-9c9c-b832f8733191 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:24:14.681504 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:24:14.713896 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e662f5e-8b4d-4ae4-957b-c75111a9b4da None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:24:14.750376 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee318f73-8b94-402f-a21c-c6167fe92336 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:24:14.751180 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee318f73-8b94-402f-a21c-c6167fe92336 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:24:14.751471 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee318f73-8b94-402f-a21c-c6167fe92336 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:24:14.751980 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee318f73-8b94-402f-a21c-c6167fe92336 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:24:14.759112 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee318f73-8b94-402f-a21c-c6167fe92336 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:24:14.760577 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee318f73-8b94-402f-a21c-c6167fe92336 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:24:14.776007 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application [None req-ee318f73-8b94-402f-a21c-c6167fe92336 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 05:24:14.776007 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:24:14.776007 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:24:14.776007 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:24:14.776007 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:24:14.776007 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:24:14.776007 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:24:14.776007 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:24:14.776007 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:24:14.776007 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:24:14.776007 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:24:14.776007 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:24:14.776007 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 25 05:24:14.776007 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 25 05:24:14.776007 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 25 05:24:14.776931 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 25 05:24:14.776931 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:24:14.776931 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:24:14.776931 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:24:14.776931 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:24:14.776931 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:24:14.776931 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:24:14.776931 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:24:14.776931 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application with Lock( Jul 25 05:24:14.776931 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:24:14.776931 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:24:14.776931 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:24:14.776931 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:24:14.776931 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:24:14.776931 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:24:14.776931 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:24:14.776931 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:24:14.778369 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 25 05:24:14.778369 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 25 05:24:14.778369 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 25 05:24:14.778369 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 25 05:24:14.778369 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 25 05:24:14.778369 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 25 05:24:14.778369 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 05:24:14.778369 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Jul 25 05:24:14.778897 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 264/531] 10.222.0.33 () {58 vars in 1282 bytes} [Mon Jul 25 05:24:14 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 339 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 25 05:24:14.778897 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-507ddbd5-84c0-47d5-9c9c-b832f8733191 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:24:14.781888 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 268/532] 10.222.0.33 () {56 vars in 1111 bytes} [Mon Jul 25 05:24:14 2022] GET /identity/v3/projects/service => generated 88 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:24:14.782943 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee318f73-8b94-402f-a21c-c6167fe92336 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:24:14.835163 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:24:14.889783 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-507ddbd5-84c0-47d5-9c9c-b832f8733191 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:24:14.893232 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee318f73-8b94-402f-a21c-c6167fe92336 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:24:14.923912 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:24:14.926309 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7698781e-c417-451b-b5a1-3921ce3e6bbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:24:14.926982 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7698781e-c417-451b-b5a1-3921ce3e6bbf None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:24:14.927269 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7698781e-c417-451b-b5a1-3921ce3e6bbf None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:24:14.927493 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7698781e-c417-451b-b5a1-3921ce3e6bbf None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:24:14.928338 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7698781e-c417-451b-b5a1-3921ce3e6bbf None admin] RBAC: Adding query filter params (role.id=200e95cafbc54269a51756ddd406082f, scope.project.id=92afba266cce4d1bad4134c351e74be3, user.id=de99e44bdf9c4030adbe9c0efa3831cb) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:24:14.928651 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7698781e-c417-451b-b5a1-3921ce3e6bbf None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:24:14.930112 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7698781e-c417-451b-b5a1-3921ce3e6bbf None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:24:14.930884 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a20f43-ce16-4f57-889b-c926dd14dbc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:24:14.931541 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a20f43-ce16-4f57-889b-c926dd14dbc1 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:24:14.931875 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a20f43-ce16-4f57-889b-c926dd14dbc1 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:24:14.932106 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a20f43-ce16-4f57-889b-c926dd14dbc1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:24:14.933146 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67a20f43-ce16-4f57-889b-c926dd14dbc1 None admin] RBAC: Adding query filter params (role.id=200e95cafbc54269a51756ddd406082f, scope.project.id=92afba266cce4d1bad4134c351e74be3, user.id=1cd482a61e8a420db45bb532e5dae6e2) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:24:14.933330 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67a20f43-ce16-4f57-889b-c926dd14dbc1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:24:14.935245 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67a20f43-ce16-4f57-889b-c926dd14dbc1 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:24:14.936994 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 265/533] 10.222.0.33 () {56 vars in 1374 bytes} [Mon Jul 25 05:24:14 2022] GET /identity/v3/role_assignments?user.id=de99e44bdf9c4030adbe9c0efa3831cb&scope.project.id=92afba266cce4d1bad4134c351e74be3&role.id=200e95cafbc54269a51756ddd406082f => generated 259 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:24:14.948028 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 269/534] 10.222.0.33 () {56 vars in 1374 bytes} [Mon Jul 25 05:24:14 2022] GET /identity/v3/role_assignments?user.id=1cd482a61e8a420db45bb532e5dae6e2&scope.project.id=92afba266cce4d1bad4134c351e74be3&role.id=200e95cafbc54269a51756ddd406082f => generated 259 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:24:14.948028 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a20f43-ce16-4f57-889b-c926dd14dbc1 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:24:14.948325 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7698781e-c417-451b-b5a1-3921ce3e6bbf None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:24:15.149717 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7698781e-c417-451b-b5a1-3921ce3e6bbf None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:24:15.153201 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a20f43-ce16-4f57-889b-c926dd14dbc1 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:24:15.176473 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:24:15.195273 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2673875e-dcf1-41ab-b5ef-6444408fe319 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:24:15.195892 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2673875e-dcf1-41ab-b5ef-6444408fe319 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:24:15.196181 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2673875e-dcf1-41ab-b5ef-6444408fe319 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:24:15.196463 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2673875e-dcf1-41ab-b5ef-6444408fe319 None admin] PATH_INFO: `/v3/projects` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:24:15.197295 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2673875e-dcf1-41ab-b5ef-6444408fe319 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:24:15.197623 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2673875e-dcf1-41ab-b5ef-6444408fe319 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:24:15.199070 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2673875e-dcf1-41ab-b5ef-6444408fe319 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:24:15.210641 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9002fe19-63dc-4cf5-b891-7a12145a1160 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:24:15.211686 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9002fe19-63dc-4cf5-b891-7a12145a1160 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:24:15.211979 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9002fe19-63dc-4cf5-b891-7a12145a1160 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:24:15.212260 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9002fe19-63dc-4cf5-b891-7a12145a1160 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:24:15.213047 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 270/535] 10.222.0.33 () {56 vars in 1148 bytes} [Mon Jul 25 05:24:14 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:24:15.219248 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2673875e-dcf1-41ab-b5ef-6444408fe319 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:24:15.235620 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9002fe19-63dc-4cf5-b891-7a12145a1160 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:24:15.237113 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9002fe19-63dc-4cf5-b891-7a12145a1160 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:24:15.263562 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application [None req-9002fe19-63dc-4cf5-b891-7a12145a1160 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 05:24:15.263562 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:24:15.263562 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:24:15.263562 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:24:15.263562 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:24:15.263562 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:24:15.263562 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:24:15.263562 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:24:15.263562 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:24:15.263562 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:24:15.263562 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:24:15.263562 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:24:15.263562 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 05:24:15.263562 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 05:24:15.263562 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 05:24:15.264477 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 05:24:15.264477 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:24:15.264477 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:24:15.264477 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:24:15.264477 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:24:15.264477 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:24:15.264477 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:24:15.264477 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:24:15.264477 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application with Lock( Jul 25 05:24:15.264477 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:24:15.264477 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:24:15.264477 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:24:15.264477 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:24:15.264477 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:24:15.264477 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:24:15.264477 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:24:15.264477 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:24:15.265379 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 05:24:15.265379 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 05:24:15.265379 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 05:24:15.265379 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Jul 25 05:24:15.266409 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 266/536] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Jul 25 05:24:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 329 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:24:15.275248 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9002fe19-63dc-4cf5-b891-7a12145a1160 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:24:15.320989 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:24:15.384559 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2673875e-dcf1-41ab-b5ef-6444408fe319 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:24:15.457918 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-813f1395-1d64-46d2-b675-0afc151dbda7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:24:15.458519 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813f1395-1d64-46d2-b675-0afc151dbda7 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:24:15.458808 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813f1395-1d64-46d2-b675-0afc151dbda7 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:24:15.459088 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813f1395-1d64-46d2-b675-0afc151dbda7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:24:15.459940 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-813f1395-1d64-46d2-b675-0afc151dbda7 None admin] RBAC: Adding query filter params (role.id=200e95cafbc54269a51756ddd406082f, scope.project.id=92afba266cce4d1bad4134c351e74be3, user.id=db57b478c25643dbaf936f90f6e17d0a) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:24:15.460265 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-813f1395-1d64-46d2-b675-0afc151dbda7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:24:15.461766 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-813f1395-1d64-46d2-b675-0afc151dbda7 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:24:15.470259 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9002fe19-63dc-4cf5-b891-7a12145a1160 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:24:15.482557 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 271/537] 10.222.0.33 () {56 vars in 1374 bytes} [Mon Jul 25 05:24:15 2022] GET /identity/v3/role_assignments?user.id=db57b478c25643dbaf936f90f6e17d0a&scope.project.id=92afba266cce4d1bad4134c351e74be3&role.id=200e95cafbc54269a51756ddd406082f => generated 259 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 05:24:15.521082 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a50e6bc7-a643-444b-95b8-c310163cbdfc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:24:15.521772 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50e6bc7-a643-444b-95b8-c310163cbdfc None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:24:15.522061 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50e6bc7-a643-444b-95b8-c310163cbdfc None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:24:15.522396 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50e6bc7-a643-444b-95b8-c310163cbdfc None admin] PATH_INFO: `/v3/domains` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:24:15.523148 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a50e6bc7-a643-444b-95b8-c310163cbdfc None admin] RBAC: Adding query filter params (name=Default) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:24:15.523510 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a50e6bc7-a643-444b-95b8-c310163cbdfc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:24:15.525130 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a50e6bc7-a643-444b-95b8-c310163cbdfc None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:24:15.544649 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 267/538] 10.222.0.33 () {56 vars in 1109 bytes} [Mon Jul 25 05:24:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:24:15.550887 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-813f1395-1d64-46d2-b675-0afc151dbda7 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:24:15.689837 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-813f1395-1d64-46d2-b675-0afc151dbda7 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:24:15.716314 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:24:15.744359 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0123a80-bafd-41ae-8f64-c0e3c5be82a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:24:15.745226 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0123a80-bafd-41ae-8f64-c0e3c5be82a1 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:24:15.745516 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0123a80-bafd-41ae-8f64-c0e3c5be82a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:24:15.745795 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0123a80-bafd-41ae-8f64-c0e3c5be82a1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:24:15.754445 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0123a80-bafd-41ae-8f64-c0e3c5be82a1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:24:15.755952 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0123a80-bafd-41ae-8f64-c0e3c5be82a1 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:24:15.781492 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application [None req-f0123a80-bafd-41ae-8f64-c0e3c5be82a1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 05:24:15.781492 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:24:15.781492 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:24:15.781492 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:24:15.781492 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:24:15.781492 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:24:15.781492 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:24:15.781492 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:24:15.781492 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:24:15.781492 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:24:15.781492 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:24:15.781492 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:24:15.781492 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 25 05:24:15.781492 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 25 05:24:15.781492 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 25 05:24:15.782423 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 25 05:24:15.782423 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:24:15.782423 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:24:15.782423 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:24:15.782423 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:24:15.782423 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:24:15.782423 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:24:15.782423 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:24:15.782423 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application with Lock( Jul 25 05:24:15.782423 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:24:15.782423 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:24:15.782423 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:24:15.782423 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:24:15.782423 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:24:15.782423 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:24:15.782423 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:24:15.782423 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:24:15.783274 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 25 05:24:15.783274 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 25 05:24:15.783274 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 25 05:24:15.783274 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 25 05:24:15.783274 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 25 05:24:15.783274 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 25 05:24:15.783274 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 05:24:15.783274 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Jul 25 05:24:15.783274 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 272/539] 10.222.0.33 () {56 vars in 1111 bytes} [Mon Jul 25 05:24:15 2022] GET /identity/v3/projects/service => generated 88 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:24:15.787660 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a50e6bc7-a643-444b-95b8-c310163cbdfc None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:24:15.807039 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3730ee2-23c9-46a5-9613-3cce23ce4fe9 None None] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:24:15.807039 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3730ee2-23c9-46a5-9613-3cce23ce4fe9 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:24:15.807340 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3730ee2-23c9-46a5-9613-3cce23ce4fe9 None None] PATH_INFO: `/` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:24:15.808706 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 273/540] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jul 25 05:24:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:24:15.816431 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8829ff-9307-463f-b786-1401b85f000b None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:24:15.816721 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8829ff-9307-463f-b786-1401b85f000b None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:24:15.817062 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8829ff-9307-463f-b786-1401b85f000b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:24:15.840829 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-bc8829ff-9307-463f-b786-1401b85f000b None None] MFA Rules not processed for user `1eae540242d94c10b0ad33960a855f08`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:24:15.840829 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-bc8829ff-9307-463f-b786-1401b85f000b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:24:15.845144 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:24:15.913846 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 274/541] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jul 25 05:24:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 05:24:15.941754 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc8829ff-9307-463f-b786-1401b85f000b None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:24:16.001494 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a50e6bc7-a643-444b-95b8-c310163cbdfc None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:24:16.078553 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce293a69-9673-4545-b39d-6698ebd03df2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:24:16.079222 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce293a69-9673-4545-b39d-6698ebd03df2 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:24:16.079511 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce293a69-9673-4545-b39d-6698ebd03df2 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:24:16.079794 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce293a69-9673-4545-b39d-6698ebd03df2 None admin] PATH_INFO: `/v3/projects` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:24:16.080597 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce293a69-9673-4545-b39d-6698ebd03df2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:24:16.080937 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce293a69-9673-4545-b39d-6698ebd03df2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:24:16.082376 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce293a69-9673-4545-b39d-6698ebd03df2 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:24:16.100928 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 268/542] 10.222.0.33 () {56 vars in 1148 bytes} [Mon Jul 25 05:24:15 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 314 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:24:16.109877 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce293a69-9673-4545-b39d-6698ebd03df2 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:24:16.128700 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc8829ff-9307-463f-b786-1401b85f000b None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:24:16.143841 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:24:16.196202 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ecc3e7b-ea47-4042-8e74-01f25416e009 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:24:16.197045 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ecc3e7b-ea47-4042-8e74-01f25416e009 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:24:16.197337 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ecc3e7b-ea47-4042-8e74-01f25416e009 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:24:16.197619 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ecc3e7b-ea47-4042-8e74-01f25416e009 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:24:16.224523 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ecc3e7b-ea47-4042-8e74-01f25416e009 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 05:24:16.232746 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application [None req-5ecc3e7b-ea47-4042-8e74-01f25416e009 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 25 05:24:16.232746 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:24:16.232746 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:24:16.232746 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:24:16.232746 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:24:16.232746 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:24:16.232746 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:24:16.232746 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:24:16.232746 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:24:16.232746 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:24:16.232746 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:24:16.232746 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:24:16.232746 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 05:24:16.232746 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 05:24:16.232746 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 05:24:16.232746 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 05:24:16.233664 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 05:24:16.233664 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 05:24:16.233664 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 05:24:16.233664 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 05:24:16.233664 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:24:16.233664 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:24:16.233664 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:24:16.233664 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:24:16.233664 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:24:16.233664 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:24:16.233664 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:24:16.233664 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application with Lock( Jul 25 05:24:16.233664 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:24:16.233664 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:24:16.233664 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:24:16.233664 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:24:16.234468 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:24:16.234468 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:24:16.234468 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:24:16.234468 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:24:16.234468 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 05:24:16.234468 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 05:24:16.234468 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 05:24:16.234468 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 05:24:16.234468 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 05:24:16.234468 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 05:24:16.234468 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 25 05:24:16.234468 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Jul 25 05:24:16.234468 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 275/543] 10.222.0.33 () {56 vars in 1102 bytes} [Mon Jul 25 05:24:15 2022] GET /identity/v3/roles/service => generated 85 bytes in 293 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 25 05:24:16.245773 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ecc3e7b-ea47-4042-8e74-01f25416e009 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:24:16.274083 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:24:16.290108 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce293a69-9673-4545-b39d-6698ebd03df2 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:24:16.359452 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e5dcd0-366d-48b4-b367-255c0282ee69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:24:16.360120 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e5dcd0-366d-48b4-b367-255c0282ee69 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:24:16.360409 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e5dcd0-366d-48b4-b367-255c0282ee69 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:24:16.360692 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e5dcd0-366d-48b4-b367-255c0282ee69 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:24:16.361581 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93e5dcd0-366d-48b4-b367-255c0282ee69 None admin] RBAC: Adding query filter params (role.id=200e95cafbc54269a51756ddd406082f, scope.project.id=92afba266cce4d1bad4134c351e74be3, user.id=b2931200f99e43d79a50a9f6e02738e6) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:24:16.361896 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93e5dcd0-366d-48b4-b367-255c0282ee69 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:24:16.363342 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93e5dcd0-366d-48b4-b367-255c0282ee69 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:24:16.369627 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 269/544] 10.222.0.33 () {56 vars in 1374 bytes} [Mon Jul 25 05:24:16 2022] GET /identity/v3/role_assignments?user.id=b2931200f99e43d79a50a9f6e02738e6&scope.project.id=92afba266cce4d1bad4134c351e74be3&role.id=200e95cafbc54269a51756ddd406082f => generated 259 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:24:16.448507 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ecc3e7b-ea47-4042-8e74-01f25416e009 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:24:16.502480 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f18cd30-e864-4319-94b7-e3bf5eba8413 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:24:16.503156 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f18cd30-e864-4319-94b7-e3bf5eba8413 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:24:16.503441 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f18cd30-e864-4319-94b7-e3bf5eba8413 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:24:16.503719 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f18cd30-e864-4319-94b7-e3bf5eba8413 None admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:24:16.504480 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f18cd30-e864-4319-94b7-e3bf5eba8413 None admin] RBAC: Adding query filter params (name=service) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:24:16.504804 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f18cd30-e864-4319-94b7-e3bf5eba8413 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:24:16.506209 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f18cd30-e864-4319-94b7-e3bf5eba8413 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:24:16.525207 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 276/545] 10.222.0.33 () {56 vars in 1103 bytes} [Mon Jul 25 05:24:16 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 280 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:24:16.526693 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e5dcd0-366d-48b4-b367-255c0282ee69 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:24:16.642485 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e5dcd0-366d-48b4-b367-255c0282ee69 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:24:16.689586 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd83266-1e08-48f9-a3b1-449e84be388b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:24:16.690768 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd83266-1e08-48f9-a3b1-449e84be388b None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:24:16.691257 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd83266-1e08-48f9-a3b1-449e84be388b None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:24:16.691733 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd83266-1e08-48f9-a3b1-449e84be388b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:24:16.703581 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dd83266-1e08-48f9-a3b1-449e84be388b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:24:16.705314 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dd83266-1e08-48f9-a3b1-449e84be388b None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:24:16.727211 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application [None req-3dd83266-1e08-48f9-a3b1-449e84be388b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 05:24:16.727211 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:24:16.727211 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:24:16.727211 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:24:16.727211 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:24:16.727211 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:24:16.727211 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:24:16.727211 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:24:16.727211 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:24:16.727211 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:24:16.727211 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:24:16.727211 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:24:16.727211 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 05:24:16.727211 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 05:24:16.727211 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 05:24:16.728099 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 05:24:16.728099 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:24:16.728099 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:24:16.728099 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:24:16.728099 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:24:16.728099 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:24:16.728099 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:24:16.728099 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:24:16.728099 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application with Lock( Jul 25 05:24:16.728099 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:24:16.728099 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:24:16.728099 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:24:16.728099 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:24:16.728099 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:24:16.728099 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:24:16.728099 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:24:16.728099 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:24:16.729044 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 05:24:16.729044 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 05:24:16.729044 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 05:24:16.729044 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Jul 25 05:24:16.731647 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 270/546] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Jul 25 05:24:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:24:16.737476 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f18cd30-e864-4319-94b7-e3bf5eba8413 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:24:16.801766 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:24:16.909696 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f18cd30-e864-4319-94b7-e3bf5eba8413 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:24:16.964003 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1534890-d30a-40cf-9ac0-c84fe07e39e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:24:16.964685 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1534890-d30a-40cf-9ac0-c84fe07e39e7 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:24:16.965063 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1534890-d30a-40cf-9ac0-c84fe07e39e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:24:16.965349 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1534890-d30a-40cf-9ac0-c84fe07e39e7 None admin] PATH_INFO: `/v3/domains` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:24:16.966177 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1534890-d30a-40cf-9ac0-c84fe07e39e7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:24:16.966494 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1534890-d30a-40cf-9ac0-c84fe07e39e7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:24:16.967881 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1534890-d30a-40cf-9ac0-c84fe07e39e7 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:24:16.984566 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 277/547] 10.222.0.33 () {56 vars in 1109 bytes} [Mon Jul 25 05:24:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:24:16.990037 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd83266-1e08-48f9-a3b1-449e84be388b None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:24:16.995772 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:24:17.126526 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd83266-1e08-48f9-a3b1-449e84be388b None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:24:17.161682 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e030834-ef90-4049-899d-343c0f22058a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:24:17.162478 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e030834-ef90-4049-899d-343c0f22058a None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:24:17.162765 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e030834-ef90-4049-899d-343c0f22058a None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:24:17.163046 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e030834-ef90-4049-899d-343c0f22058a None admin] PATH_INFO: `/v3/users/swift` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:24:17.171293 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e030834-ef90-4049-899d-343c0f22058a None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:24:17.172749 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e030834-ef90-4049-899d-343c0f22058a None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:24:17.187606 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application [None req-7e030834-ef90-4049-899d-343c0f22058a None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 25 05:24:17.187606 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:24:17.187606 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:24:17.187606 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:24:17.187606 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:24:17.187606 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:24:17.187606 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:24:17.187606 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:24:17.187606 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:24:17.187606 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:24:17.187606 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:24:17.187606 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:24:17.187606 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 25 05:24:17.187606 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 25 05:24:17.187606 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 25 05:24:17.187606 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 25 05:24:17.188585 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:24:17.188585 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:24:17.188585 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 25 05:24:17.188585 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 05:24:17.188585 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 25 05:24:17.188585 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 05:24:17.188585 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:24:17.188585 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:24:17.188585 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:24:17.188585 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:24:17.188585 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:24:17.188585 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application with Lock( Jul 25 05:24:17.188585 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:24:17.188585 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:24:17.188585 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:24:17.188585 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:24:17.188585 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:24:17.188585 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:24:17.190914 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:24:17.190914 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:24:17.190914 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 25 05:24:17.190914 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 25 05:24:17.190914 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 25 05:24:17.190914 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 25 05:24:17.190914 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 25 05:24:17.190914 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 25 05:24:17.190914 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 25 05:24:17.190914 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Jul 25 05:24:17.190914 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 271/548] 10.222.0.33 () {56 vars in 1096 bytes} [Mon Jul 25 05:24:16 2022] GET /identity/v3/users/swift => generated 83 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:24:17.194710 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1534890-d30a-40cf-9ac0-c84fe07e39e7 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:24:17.350486 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:24:17.363513 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1534890-d30a-40cf-9ac0-c84fe07e39e7 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:24:17.398980 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8e9e40-8d48-4b25-8488-b02fd3ed8b02 None None] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:24:17.399221 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8e9e40-8d48-4b25-8488-b02fd3ed8b02 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:24:17.399468 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8e9e40-8d48-4b25-8488-b02fd3ed8b02 None None] PATH_INFO: `/` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:24:17.404802 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 272/549] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jul 25 05:24:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 25 05:24:17.406420 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f42659-cf3a-4161-9373-bd5b77ba29f3 None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:24:17.406708 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f42659-cf3a-4161-9373-bd5b77ba29f3 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:24:17.406991 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f42659-cf3a-4161-9373-bd5b77ba29f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:24:17.425487 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-194f5d8e-24be-4a0a-bd03-f2880d4b3ee9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:24:17.426444 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194f5d8e-24be-4a0a-bd03-f2880d4b3ee9 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:24:17.426577 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-10f42659-cf3a-4161-9373-bd5b77ba29f3 None None] MFA Rules not processed for user `1eae540242d94c10b0ad33960a855f08`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:24:17.427037 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194f5d8e-24be-4a0a-bd03-f2880d4b3ee9 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:24:17.427489 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194f5d8e-24be-4a0a-bd03-f2880d4b3ee9 None admin] PATH_INFO: `/v3/users` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:24:17.427795 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-10f42659-cf3a-4161-9373-bd5b77ba29f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:24:17.428698 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-194f5d8e-24be-4a0a-bd03-f2880d4b3ee9 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:24:17.429227 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-194f5d8e-24be-4a0a-bd03-f2880d4b3ee9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:24:17.430827 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-194f5d8e-24be-4a0a-bd03-f2880d4b3ee9 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:24:17.462280 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 278/550] 10.222.0.33 () {56 vars in 1135 bytes} [Mon Jul 25 05:24:17 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:24:17.468637 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-194f5d8e-24be-4a0a-bd03-f2880d4b3ee9 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:24:17.502379 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:24:17.515043 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 273/551] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jul 25 05:24:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 25 05:24:17.541246 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f42659-cf3a-4161-9373-bd5b77ba29f3 None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:24:17.629080 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-194f5d8e-24be-4a0a-bd03-f2880d4b3ee9 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:24:17.679939 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b75b1a9-2af1-4d3f-84fc-84ab4e51141a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:24:17.680536 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b75b1a9-2af1-4d3f-84fc-84ab4e51141a None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:24:17.680958 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b75b1a9-2af1-4d3f-84fc-84ab4e51141a None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:24:17.681242 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b75b1a9-2af1-4d3f-84fc-84ab4e51141a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:24:17.689706 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f42659-cf3a-4161-9373-bd5b77ba29f3 None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:24:17.704901 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:24:17.730767 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba89ce4-9f67-44da-ae0b-1d63c34a4a20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:24:17.731563 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba89ce4-9f67-44da-ae0b-1d63c34a4a20 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:24:17.731844 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba89ce4-9f67-44da-ae0b-1d63c34a4a20 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:24:17.732223 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba89ce4-9f67-44da-ae0b-1d63c34a4a20 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:24:17.745210 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bba89ce4-9f67-44da-ae0b-1d63c34a4a20 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 05:24:17.756767 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application [None req-bba89ce4-9f67-44da-ae0b-1d63c34a4a20 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 25 05:24:17.756767 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:24:17.756767 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:24:17.756767 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:24:17.756767 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:24:17.756767 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:24:17.756767 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:24:17.756767 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:24:17.756767 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:24:17.756767 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:24:17.756767 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:24:17.756767 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:24:17.756767 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 05:24:17.756767 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 05:24:17.756767 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 05:24:17.756767 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 05:24:17.757871 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 05:24:17.757871 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 05:24:17.757871 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 05:24:17.757871 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 05:24:17.757871 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:24:17.757871 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:24:17.757871 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:24:17.757871 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:24:17.757871 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:24:17.757871 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:24:17.757871 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:24:17.757871 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application with Lock( Jul 25 05:24:17.757871 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:24:17.757871 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:24:17.757871 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:24:17.757871 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:24:17.758982 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:24:17.758982 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:24:17.758982 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:24:17.758982 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:24:17.758982 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 05:24:17.758982 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 05:24:17.758982 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 05:24:17.758982 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 05:24:17.758982 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 05:24:17.758982 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 05:24:17.758982 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 25 05:24:17.758982 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Jul 25 05:24:17.758982 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 274/552] 10.222.0.33 () {56 vars in 1102 bytes} [Mon Jul 25 05:24:17 2022] GET /identity/v3/roles/service => generated 85 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:24:17.761628 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e60f39-2f7f-4703-b78f-0eaf4cc743df None None] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:24:17.761628 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e60f39-2f7f-4703-b78f-0eaf4cc743df None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:24:17.761628 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e60f39-2f7f-4703-b78f-0eaf4cc743df None None] PATH_INFO: `/` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:24:17.762790 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b75b1a9-2af1-4d3f-84fc-84ab4e51141a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:24:17.768800 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 275/553] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jul 25 05:24:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:24:17.768800 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e60f39-2f7f-4703-b78f-0eaf4cc743df None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:24:17.769096 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b75b1a9-2af1-4d3f-84fc-84ab4e51141a None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:24:17.843690 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:24:17.891889 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 279/554] 10.222.0.33 () {58 vars in 1282 bytes} [Mon Jul 25 05:24:17 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 419 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 25 05:24:17.893681 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938db2dc-2ef3-4484-9981-8a391415cc5f None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:24:17.894058 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938db2dc-2ef3-4484-9981-8a391415cc5f None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:24:17.894428 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938db2dc-2ef3-4484-9981-8a391415cc5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:24:17.911924 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e60f39-2f7f-4703-b78f-0eaf4cc743df None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:24:17.912449 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-938db2dc-2ef3-4484-9981-8a391415cc5f None None] MFA Rules not processed for user `1eae540242d94c10b0ad33960a855f08`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:24:17.913792 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-938db2dc-2ef3-4484-9981-8a391415cc5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:24:17.960169 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-770174bd-1e3a-4252-bfff-2b6de394daec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:24:17.961157 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770174bd-1e3a-4252-bfff-2b6de394daec None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:24:17.961620 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770174bd-1e3a-4252-bfff-2b6de394daec None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:24:17.962067 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770174bd-1e3a-4252-bfff-2b6de394daec None admin] PATH_INFO: `/v3/roles` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:24:17.963552 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-770174bd-1e3a-4252-bfff-2b6de394daec None admin] RBAC: Adding query filter params (name=service) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:24:17.964041 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-770174bd-1e3a-4252-bfff-2b6de394daec None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:24:17.966217 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-770174bd-1e3a-4252-bfff-2b6de394daec None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:24:17.967160 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:24:17.979833 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 276/555] 10.222.0.33 () {56 vars in 1103 bytes} [Mon Jul 25 05:24:17 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:24:17.981661 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-770174bd-1e3a-4252-bfff-2b6de394daec None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:24:17.997270 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 280/556] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jul 25 05:24:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 05:24:17.998975 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71f5a76-ab48-4b10-ac6a-27dbb8e3db93 None None] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:24:17.999441 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71f5a76-ab48-4b10-ac6a-27dbb8e3db93 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:24:17.999961 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71f5a76-ab48-4b10-ac6a-27dbb8e3db93 None None] PATH_INFO: `/` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:24:18.002259 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 281/557] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jul 25 05:24:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:24:18.003517 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d71f5a76-ab48-4b10-ac6a-27dbb8e3db93 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:24:18.099821 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-770174bd-1e3a-4252-bfff-2b6de394daec None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:24:18.112929 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d71f5a76-ab48-4b10-ac6a-27dbb8e3db93 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:24:18.114720 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:24:18.137612 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-132e2e01-d424-49c1-b023-48c27432dfaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:24:18.138401 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132e2e01-d424-49c1-b023-48c27432dfaf None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:24:18.138683 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132e2e01-d424-49c1-b023-48c27432dfaf None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:24:18.138963 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132e2e01-d424-49c1-b023-48c27432dfaf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:24:18.149606 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e995ce2-85ff-4d90-8b72-f7c261a0188e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:24:18.150243 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-132e2e01-d424-49c1-b023-48c27432dfaf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:24:18.150435 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e995ce2-85ff-4d90-8b72-f7c261a0188e None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:24:18.150683 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e995ce2-85ff-4d90-8b72-f7c261a0188e None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:24:18.150965 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e995ce2-85ff-4d90-8b72-f7c261a0188e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:24:18.151666 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-132e2e01-d424-49c1-b023-48c27432dfaf None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:24:18.162077 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e995ce2-85ff-4d90-8b72-f7c261a0188e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:24:18.163518 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e995ce2-85ff-4d90-8b72-f7c261a0188e None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:24:18.169413 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application [None req-132e2e01-d424-49c1-b023-48c27432dfaf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 05:24:18.169413 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:24:18.169413 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:24:18.169413 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:24:18.169413 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:24:18.169413 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:24:18.169413 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:24:18.169413 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:24:18.169413 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:24:18.169413 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:24:18.169413 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:24:18.169413 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:24:18.169413 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 05:24:18.169413 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 05:24:18.169413 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 05:24:18.170302 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 05:24:18.170302 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:24:18.170302 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:24:18.170302 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:24:18.170302 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:24:18.170302 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:24:18.170302 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:24:18.170302 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:24:18.170302 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application with Lock( Jul 25 05:24:18.170302 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:24:18.170302 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:24:18.170302 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:24:18.170302 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:24:18.170302 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:24:18.170302 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:24:18.170302 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:24:18.170302 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:24:18.171204 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 05:24:18.171204 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 05:24:18.171204 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 05:24:18.171204 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Jul 25 05:24:18.171204 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 277/558] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Jul 25 05:24:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:24:18.172515 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee47ee3-d75c-46d4-99af-244cd49a42ea None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:24:18.172515 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee47ee3-d75c-46d4-99af-244cd49a42ea None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:24:18.172808 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee47ee3-d75c-46d4-99af-244cd49a42ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:24:18.184904 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application [None req-3e995ce2-85ff-4d90-8b72-f7c261a0188e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 05:24:18.184904 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:24:18.184904 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:24:18.184904 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:24:18.184904 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:24:18.184904 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:24:18.184904 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:24:18.184904 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:24:18.184904 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:24:18.184904 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:24:18.184904 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:24:18.184904 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:24:18.184904 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 05:24:18.184904 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 05:24:18.184904 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 05:24:18.186452 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 05:24:18.186452 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:24:18.186452 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:24:18.186452 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:24:18.186452 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:24:18.186452 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:24:18.186452 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:24:18.186452 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:24:18.186452 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application with Lock( Jul 25 05:24:18.186452 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:24:18.186452 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:24:18.186452 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:24:18.186452 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:24:18.186452 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:24:18.186452 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:24:18.186452 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:24:18.186452 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:24:18.187583 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 05:24:18.187583 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 05:24:18.187583 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 05:24:18.187583 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Jul 25 05:24:18.189433 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 282/559] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Jul 25 05:24:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:24:18.189706 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-5ee47ee3-d75c-46d4-99af-244cd49a42ea None None] MFA Rules not processed for user `1eae540242d94c10b0ad33960a855f08`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:24:18.190510 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e995ce2-85ff-4d90-8b72-f7c261a0188e None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:24:18.190962 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-5ee47ee3-d75c-46d4-99af-244cd49a42ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:24:18.223795 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:24:18.276800 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 278/560] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jul 25 05:24:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 05:24:18.276800 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee47ee3-d75c-46d4-99af-244cd49a42ea None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:24:18.307017 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e995ce2-85ff-4d90-8b72-f7c261a0188e None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:24:18.341836 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb132a15-405c-4dab-bb86-a274de840737 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:24:18.342641 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb132a15-405c-4dab-bb86-a274de840737 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:24:18.342939 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb132a15-405c-4dab-bb86-a274de840737 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:24:18.343219 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb132a15-405c-4dab-bb86-a274de840737 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:24:18.353447 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb132a15-405c-4dab-bb86-a274de840737 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 05:24:18.361773 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application [None req-bb132a15-405c-4dab-bb86-a274de840737 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 25 05:24:18.361773 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:24:18.361773 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:24:18.361773 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:24:18.361773 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:24:18.361773 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:24:18.361773 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:24:18.361773 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:24:18.361773 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:24:18.361773 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:24:18.361773 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:24:18.361773 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:24:18.361773 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 05:24:18.361773 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 05:24:18.361773 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 05:24:18.361773 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 05:24:18.362878 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 05:24:18.362878 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 05:24:18.362878 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 05:24:18.362878 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 05:24:18.362878 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:24:18.362878 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:24:18.362878 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:24:18.362878 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:24:18.362878 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:24:18.362878 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:24:18.362878 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:24:18.362878 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application with Lock( Jul 25 05:24:18.362878 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:24:18.362878 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:24:18.362878 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:24:18.362878 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:24:18.363803 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:24:18.363803 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:24:18.363803 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:24:18.363803 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:24:18.363803 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 05:24:18.363803 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 05:24:18.363803 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 05:24:18.363803 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 05:24:18.363803 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 05:24:18.363803 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 05:24:18.363803 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 25 05:24:18.363803 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Jul 25 05:24:18.363803 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 283/561] 10.222.0.33 () {56 vars in 1102 bytes} [Mon Jul 25 05:24:18 2022] GET /identity/v3/roles/service => generated 85 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:24:18.363803 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb132a15-405c-4dab-bb86-a274de840737 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:24:18.383730 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:24:18.391624 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee47ee3-d75c-46d4-99af-244cd49a42ea None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:24:18.426988 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-494e5931-d913-4a53-aead-f7a8dbcbc109 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:24:18.427656 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494e5931-d913-4a53-aead-f7a8dbcbc109 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:24:18.427941 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494e5931-d913-4a53-aead-f7a8dbcbc109 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:24:18.428220 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494e5931-d913-4a53-aead-f7a8dbcbc109 None admin] PATH_INFO: `/v3/domains` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:24:18.428980 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-494e5931-d913-4a53-aead-f7a8dbcbc109 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:24:18.429297 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-494e5931-d913-4a53-aead-f7a8dbcbc109 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:24:18.430675 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-494e5931-d913-4a53-aead-f7a8dbcbc109 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:24:18.447159 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 279/562] 10.222.0.33 () {56 vars in 1109 bytes} [Mon Jul 25 05:24:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:24:18.447159 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-494e5931-d913-4a53-aead-f7a8dbcbc109 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:24:18.472765 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:24:18.485499 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb132a15-405c-4dab-bb86-a274de840737 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:24:18.522156 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78938e1c-3be2-4f40-b4a4-26a3592df70c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:24:18.522833 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78938e1c-3be2-4f40-b4a4-26a3592df70c None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:24:18.523122 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78938e1c-3be2-4f40-b4a4-26a3592df70c None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:24:18.523404 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78938e1c-3be2-4f40-b4a4-26a3592df70c None admin] PATH_INFO: `/v3/domains` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:24:18.524141 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78938e1c-3be2-4f40-b4a4-26a3592df70c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:24:18.524454 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78938e1c-3be2-4f40-b4a4-26a3592df70c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:24:18.525895 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78938e1c-3be2-4f40-b4a4-26a3592df70c None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:24:18.539657 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 284/563] 10.222.0.33 () {56 vars in 1109 bytes} [Mon Jul 25 05:24:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:24:18.540946 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78938e1c-3be2-4f40-b4a4-26a3592df70c None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:24:18.557896 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-494e5931-d913-4a53-aead-f7a8dbcbc109 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:24:18.594653 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc1f3ab-4b81-4f97-8fa2-795aceed5fb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:24:18.595450 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc1f3ab-4b81-4f97-8fa2-795aceed5fb1 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:24:18.595741 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc1f3ab-4b81-4f97-8fa2-795aceed5fb1 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:24:18.596020 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc1f3ab-4b81-4f97-8fa2-795aceed5fb1 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:24:18.607128 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc1f3ab-4b81-4f97-8fa2-795aceed5fb1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 05:24:18.615014 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application [None req-edc1f3ab-4b81-4f97-8fa2-795aceed5fb1 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 25 05:24:18.615014 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:24:18.615014 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:24:18.615014 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:24:18.615014 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:24:18.615014 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:24:18.615014 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:24:18.615014 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:24:18.615014 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:24:18.615014 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:24:18.615014 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:24:18.615014 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:24:18.615014 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 05:24:18.615014 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 05:24:18.615014 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 05:24:18.615014 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 05:24:18.616408 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 05:24:18.616408 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 05:24:18.616408 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 05:24:18.616408 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 05:24:18.616408 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:24:18.616408 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:24:18.616408 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:24:18.616408 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:24:18.616408 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:24:18.616408 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:24:18.616408 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:24:18.616408 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application with Lock( Jul 25 05:24:18.616408 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:24:18.616408 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:24:18.616408 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:24:18.616408 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:24:18.618047 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:24:18.618047 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:24:18.618047 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:24:18.618047 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:24:18.618047 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 05:24:18.618047 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 05:24:18.618047 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 05:24:18.618047 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 05:24:18.618047 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 05:24:18.618047 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 05:24:18.618047 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 25 05:24:18.618047 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Jul 25 05:24:18.618047 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 280/564] 10.222.0.33 () {56 vars in 1102 bytes} [Mon Jul 25 05:24:18 2022] GET /identity/v3/roles/service => generated 85 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:24:18.618047 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc1f3ab-4b81-4f97-8fa2-795aceed5fb1 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:24:18.642187 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:24:18.651312 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78938e1c-3be2-4f40-b4a4-26a3592df70c None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:24:18.694050 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-184ace13-d99e-467a-83c4-3e7c744ce6ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:24:18.694970 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184ace13-d99e-467a-83c4-3e7c744ce6ac None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:24:18.695256 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184ace13-d99e-467a-83c4-3e7c744ce6ac None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:24:18.695560 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184ace13-d99e-467a-83c4-3e7c744ce6ac None admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:24:18.697164 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-184ace13-d99e-467a-83c4-3e7c744ce6ac None admin] RBAC: Adding query filter params (name=service) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:24:18.697480 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-184ace13-d99e-467a-83c4-3e7c744ce6ac None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:24:18.698842 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-184ace13-d99e-467a-83c4-3e7c744ce6ac None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:24:18.714345 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 285/565] 10.222.0.33 () {56 vars in 1103 bytes} [Mon Jul 25 05:24:18 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:24:18.714345 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-184ace13-d99e-467a-83c4-3e7c744ce6ac None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:24:18.727294 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc1f3ab-4b81-4f97-8fa2-795aceed5fb1 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:24:18.734980 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:24:18.766695 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79783f6a-b8a7-4d0b-8881-0e5262730bc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:24:18.767561 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79783f6a-b8a7-4d0b-8881-0e5262730bc9 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:24:18.767846 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79783f6a-b8a7-4d0b-8881-0e5262730bc9 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:24:18.768118 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79783f6a-b8a7-4d0b-8881-0e5262730bc9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:24:18.774971 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79783f6a-b8a7-4d0b-8881-0e5262730bc9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:24:18.776403 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79783f6a-b8a7-4d0b-8881-0e5262730bc9 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:24:18.791124 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application [None req-79783f6a-b8a7-4d0b-8881-0e5262730bc9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 05:24:18.791124 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:24:18.791124 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:24:18.791124 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:24:18.791124 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:24:18.791124 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:24:18.791124 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:24:18.791124 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:24:18.791124 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:24:18.791124 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:24:18.791124 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:24:18.791124 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:24:18.791124 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 25 05:24:18.791124 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 25 05:24:18.791124 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 25 05:24:18.792118 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 25 05:24:18.792118 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:24:18.792118 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:24:18.792118 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:24:18.792118 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:24:18.792118 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:24:18.792118 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:24:18.792118 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:24:18.792118 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application with Lock( Jul 25 05:24:18.792118 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:24:18.792118 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:24:18.792118 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:24:18.792118 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:24:18.792118 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:24:18.792118 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:24:18.792118 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:24:18.792118 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:24:18.793874 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 25 05:24:18.793874 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 25 05:24:18.793874 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 25 05:24:18.793874 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 25 05:24:18.793874 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 25 05:24:18.793874 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 25 05:24:18.793874 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 05:24:18.793874 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Jul 25 05:24:18.793874 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 281/566] 10.222.0.33 () {56 vars in 1111 bytes} [Mon Jul 25 05:24:18 2022] GET /identity/v3/projects/service => generated 88 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:24:18.793874 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79783f6a-b8a7-4d0b-8881-0e5262730bc9 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:24:18.835859 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-184ace13-d99e-467a-83c4-3e7c744ce6ac None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:24:18.869742 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8094c1b-47ca-4151-bcd0-4d1f6b8ccb48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:24:18.870535 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8094c1b-47ca-4151-bcd0-4d1f6b8ccb48 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:24:18.870822 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8094c1b-47ca-4151-bcd0-4d1f6b8ccb48 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:24:18.871126 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8094c1b-47ca-4151-bcd0-4d1f6b8ccb48 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:24:18.873652 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:24:18.878787 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8094c1b-47ca-4151-bcd0-4d1f6b8ccb48 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:24:18.880233 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8094c1b-47ca-4151-bcd0-4d1f6b8ccb48 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:24:18.895245 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application [None req-a8094c1b-47ca-4151-bcd0-4d1f6b8ccb48 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jul 25 05:24:18.895245 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:24:18.895245 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:24:18.895245 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:24:18.895245 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:24:18.895245 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:24:18.895245 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:24:18.895245 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:24:18.895245 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:24:18.895245 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:24:18.895245 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:24:18.895245 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:24:18.895245 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 25 05:24:18.895245 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 25 05:24:18.895245 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 25 05:24:18.895245 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 25 05:24:18.896225 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:24:18.896225 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:24:18.896225 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 25 05:24:18.896225 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 05:24:18.896225 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 25 05:24:18.896225 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 05:24:18.896225 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:24:18.896225 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:24:18.896225 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:24:18.896225 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:24:18.896225 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:24:18.896225 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application with Lock( Jul 25 05:24:18.896225 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:24:18.896225 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:24:18.896225 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:24:18.896225 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:24:18.896225 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:24:18.896225 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:24:18.897224 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:24:18.897224 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:24:18.897224 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 25 05:24:18.897224 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 25 05:24:18.897224 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 25 05:24:18.897224 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 25 05:24:18.897224 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 25 05:24:18.897224 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 25 05:24:18.897224 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jul 25 05:24:18.897224 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Jul 25 05:24:18.897224 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 286/567] 10.222.0.33 () {56 vars in 1099 bytes} [Mon Jul 25 05:24:18 2022] GET /identity/v3/users/glance => generated 84 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:24:18.897841 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79783f6a-b8a7-4d0b-8881-0e5262730bc9 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:24:18.897918 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8094c1b-47ca-4151-bcd0-4d1f6b8ccb48 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:24:18.933502 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61af51c8-dbe2-460e-8c98-e8a4fe5144d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:24:18.934318 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61af51c8-dbe2-460e-8c98-e8a4fe5144d6 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:24:18.934601 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61af51c8-dbe2-460e-8c98-e8a4fe5144d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:24:18.934879 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61af51c8-dbe2-460e-8c98-e8a4fe5144d6 None admin] PATH_INFO: `/v3/roles` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:24:18.935622 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61af51c8-dbe2-460e-8c98-e8a4fe5144d6 None admin] RBAC: Adding query filter params (name=service) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:24:18.935928 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61af51c8-dbe2-460e-8c98-e8a4fe5144d6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:24:18.937321 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61af51c8-dbe2-460e-8c98-e8a4fe5144d6 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:24:18.945488 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 282/568] 10.222.0.33 () {56 vars in 1103 bytes} [Mon Jul 25 05:24:18 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:24:18.946276 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61af51c8-dbe2-460e-8c98-e8a4fe5144d6 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:24:18.970700 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:24:18.999946 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8094c1b-47ca-4151-bcd0-4d1f6b8ccb48 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:24:19.032923 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5087dae6-9635-43c3-a641-0e1050c8373a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:24:19.033641 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5087dae6-9635-43c3-a641-0e1050c8373a None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:24:19.033924 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5087dae6-9635-43c3-a641-0e1050c8373a None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:24:19.034202 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5087dae6-9635-43c3-a641-0e1050c8373a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:24:19.045469 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5087dae6-9635-43c3-a641-0e1050c8373a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:24:19.046294 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61af51c8-dbe2-460e-8c98-e8a4fe5144d6 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:24:19.046927 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5087dae6-9635-43c3-a641-0e1050c8373a None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:24:19.064295 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application [None req-5087dae6-9635-43c3-a641-0e1050c8373a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 05:24:19.064295 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:24:19.064295 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:24:19.064295 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:24:19.064295 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:24:19.064295 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:24:19.064295 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:24:19.064295 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:24:19.064295 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:24:19.064295 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:24:19.064295 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:24:19.064295 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:24:19.064295 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 05:24:19.064295 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 05:24:19.064295 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 05:24:19.065219 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 05:24:19.065219 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:24:19.065219 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:24:19.065219 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:24:19.065219 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:24:19.065219 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:24:19.065219 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:24:19.065219 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:24:19.065219 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application with Lock( Jul 25 05:24:19.065219 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:24:19.065219 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:24:19.065219 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:24:19.065219 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:24:19.065219 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:24:19.065219 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:24:19.065219 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:24:19.065219 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:24:19.066129 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 05:24:19.066129 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 05:24:19.066129 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 05:24:19.066129 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Jul 25 05:24:19.066129 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 287/569] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Jul 25 05:24:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:24:19.067164 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca72dc8-de06-4d3e-ad76-634f42b9e2ed None None] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:24:19.067272 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca72dc8-de06-4d3e-ad76-634f42b9e2ed None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:24:19.067524 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca72dc8-de06-4d3e-ad76-634f42b9e2ed None None] PATH_INFO: `/` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:24:19.068943 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 288/570] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jul 25 05:24:19 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:24:19.069503 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca72dc8-de06-4d3e-ad76-634f42b9e2ed None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:24:19.083270 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e41c52d2-03bb-4883-a131-f4428d430ac3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:24:19.083927 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41c52d2-03bb-4883-a131-f4428d430ac3 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:24:19.084208 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41c52d2-03bb-4883-a131-f4428d430ac3 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:24:19.084481 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41c52d2-03bb-4883-a131-f4428d430ac3 None admin] PATH_INFO: `/v3/projects` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:24:19.085308 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e41c52d2-03bb-4883-a131-f4428d430ac3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:24:19.085619 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e41c52d2-03bb-4883-a131-f4428d430ac3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:24:19.087046 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e41c52d2-03bb-4883-a131-f4428d430ac3 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:24:19.094096 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:24:19.099366 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 283/571] 10.222.0.33 () {56 vars in 1148 bytes} [Mon Jul 25 05:24:18 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:24:19.099366 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e41c52d2-03bb-4883-a131-f4428d430ac3 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:24:19.176918 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca72dc8-de06-4d3e-ad76-634f42b9e2ed None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:24:19.198272 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e41c52d2-03bb-4883-a131-f4428d430ac3 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:24:19.200676 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:24:19.211776 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce3dee11-951f-4d86-afe9-82a8811e512d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:24:19.212438 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3dee11-951f-4d86-afe9-82a8811e512d None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:24:19.212720 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3dee11-951f-4d86-afe9-82a8811e512d None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:24:19.213025 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3dee11-951f-4d86-afe9-82a8811e512d None admin] PATH_INFO: `/v3/users` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:24:19.213858 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce3dee11-951f-4d86-afe9-82a8811e512d None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:24:19.214161 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce3dee11-951f-4d86-afe9-82a8811e512d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:24:19.216234 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce3dee11-951f-4d86-afe9-82a8811e512d None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:24:19.233684 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 289/572] 10.222.0.33 () {56 vars in 1137 bytes} [Mon Jul 25 05:24:19 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:24:19.234467 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce3dee11-951f-4d86-afe9-82a8811e512d None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:24:19.239455 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c691e411-bed4-4b09-aa96-ee08bdeed51e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:24:19.240265 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c691e411-bed4-4b09-aa96-ee08bdeed51e None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:24:19.240552 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c691e411-bed4-4b09-aa96-ee08bdeed51e None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:24:19.240860 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c691e411-bed4-4b09-aa96-ee08bdeed51e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:24:19.252083 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c691e411-bed4-4b09-aa96-ee08bdeed51e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:24:19.253561 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c691e411-bed4-4b09-aa96-ee08bdeed51e None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:24:19.271188 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application [None req-c691e411-bed4-4b09-aa96-ee08bdeed51e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 05:24:19.271188 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:24:19.271188 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:24:19.271188 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:24:19.271188 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:24:19.271188 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:24:19.271188 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:24:19.271188 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:24:19.271188 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:24:19.271188 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:24:19.271188 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:24:19.271188 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:24:19.271188 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 05:24:19.271188 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 05:24:19.271188 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 05:24:19.272079 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 05:24:19.272079 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:24:19.272079 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:24:19.272079 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:24:19.272079 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:24:19.272079 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:24:19.272079 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:24:19.272079 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:24:19.272079 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application with Lock( Jul 25 05:24:19.272079 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:24:19.272079 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:24:19.272079 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:24:19.272079 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:24:19.272079 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:24:19.272079 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:24:19.272079 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:24:19.272079 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:24:19.273002 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 05:24:19.273002 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 05:24:19.273002 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 05:24:19.273002 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Jul 25 05:24:19.273002 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 284/573] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Jul 25 05:24:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:24:19.273697 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da19a99-0195-4a93-becf-bd570c654606 None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:24:19.273864 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da19a99-0195-4a93-becf-bd570c654606 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:24:19.274123 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da19a99-0195-4a93-becf-bd570c654606 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:24:19.291342 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-6da19a99-0195-4a93-becf-bd570c654606 None None] MFA Rules not processed for user `1eae540242d94c10b0ad33960a855f08`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:24:19.292546 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-6da19a99-0195-4a93-becf-bd570c654606 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:24:19.335765 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce3dee11-951f-4d86-afe9-82a8811e512d None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:24:19.338351 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:24:19.372804 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 285/574] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jul 25 05:24:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 05:24:19.372804 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da19a99-0195-4a93-becf-bd570c654606 None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:24:19.373082 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb3dad59-c3d1-4fba-aa6d-2859e3b5d52f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:24:19.373082 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3dad59-c3d1-4fba-aa6d-2859e3b5d52f None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:24:19.373082 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3dad59-c3d1-4fba-aa6d-2859e3b5d52f None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:24:19.373082 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3dad59-c3d1-4fba-aa6d-2859e3b5d52f None admin] PATH_INFO: `/v3/domains` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:24:19.373396 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb3dad59-c3d1-4fba-aa6d-2859e3b5d52f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:24:19.373677 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb3dad59-c3d1-4fba-aa6d-2859e3b5d52f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:24:19.382547 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb3dad59-c3d1-4fba-aa6d-2859e3b5d52f None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:24:19.388718 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 290/575] 10.222.0.33 () {56 vars in 1109 bytes} [Mon Jul 25 05:24:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:24:19.389546 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb3dad59-c3d1-4fba-aa6d-2859e3b5d52f None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:24:19.435045 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:24:19.457615 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da19a99-0195-4a93-becf-bd570c654606 None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:24:19.492164 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e35155-0674-4e3d-908f-9f16b05eb6a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:24:19.492420 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb3dad59-c3d1-4fba-aa6d-2859e3b5d52f None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:24:19.492814 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e35155-0674-4e3d-908f-9f16b05eb6a9 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:24:19.493098 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e35155-0674-4e3d-908f-9f16b05eb6a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:24:19.493375 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e35155-0674-4e3d-908f-9f16b05eb6a9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:24:19.494538 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62e35155-0674-4e3d-908f-9f16b05eb6a9 None admin] RBAC: Adding query filter params (role.id=200e95cafbc54269a51756ddd406082f, scope.project.id=92afba266cce4d1bad4134c351e74be3, user.id=eb135a8d5d7444cfbfa336115ba2ba44) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:24:19.494854 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62e35155-0674-4e3d-908f-9f16b05eb6a9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:24:19.496861 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62e35155-0674-4e3d-908f-9f16b05eb6a9 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:24:19.502336 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 286/576] 10.222.0.33 () {56 vars in 1374 bytes} [Mon Jul 25 05:24:19 2022] GET /identity/v3/role_assignments?user.id=eb135a8d5d7444cfbfa336115ba2ba44&scope.project.id=92afba266cce4d1bad4134c351e74be3&role.id=200e95cafbc54269a51756ddd406082f => generated 259 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:24:19.503123 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e35155-0674-4e3d-908f-9f16b05eb6a9 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:24:19.533241 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f672b20d-7e39-4748-8e40-2607e0a35a17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:24:19.534270 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f672b20d-7e39-4748-8e40-2607e0a35a17 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:24:19.534650 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f672b20d-7e39-4748-8e40-2607e0a35a17 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:24:19.535015 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f672b20d-7e39-4748-8e40-2607e0a35a17 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:24:19.547608 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f672b20d-7e39-4748-8e40-2607e0a35a17 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:24:19.549040 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f672b20d-7e39-4748-8e40-2607e0a35a17 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:24:19.555890 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:24:19.571463 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application [None req-f672b20d-7e39-4748-8e40-2607e0a35a17 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 05:24:19.571463 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:24:19.571463 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:24:19.571463 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:24:19.571463 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:24:19.571463 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:24:19.571463 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:24:19.571463 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:24:19.571463 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:24:19.571463 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:24:19.571463 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:24:19.571463 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:24:19.571463 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 05:24:19.571463 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 05:24:19.571463 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 05:24:19.572638 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 05:24:19.572638 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:24:19.572638 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:24:19.572638 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:24:19.572638 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:24:19.572638 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:24:19.572638 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:24:19.572638 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:24:19.572638 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application with Lock( Jul 25 05:24:19.572638 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:24:19.572638 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:24:19.572638 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:24:19.572638 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:24:19.572638 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:24:19.572638 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:24:19.572638 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:24:19.572638 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:24:19.579299 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 05:24:19.579299 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 05:24:19.579299 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 05:24:19.579299 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Jul 25 05:24:19.579299 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 291/577] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Jul 25 05:24:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:24:19.579299 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f672b20d-7e39-4748-8e40-2607e0a35a17 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:24:19.638993 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e35155-0674-4e3d-908f-9f16b05eb6a9 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:24:19.680273 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7954d33f-1ab2-43ec-b765-27a3db6dc70d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:24:19.680982 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7954d33f-1ab2-43ec-b765-27a3db6dc70d None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:24:19.681278 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7954d33f-1ab2-43ec-b765-27a3db6dc70d None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:24:19.681562 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7954d33f-1ab2-43ec-b765-27a3db6dc70d None admin] PATH_INFO: `/v3/domains` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:24:19.681988 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:24:19.682301 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7954d33f-1ab2-43ec-b765-27a3db6dc70d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:24:19.682615 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7954d33f-1ab2-43ec-b765-27a3db6dc70d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:24:19.684963 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7954d33f-1ab2-43ec-b765-27a3db6dc70d None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:24:19.690020 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f672b20d-7e39-4748-8e40-2607e0a35a17 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:24:19.697067 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 287/578] 10.222.0.33 () {56 vars in 1109 bytes} [Mon Jul 25 05:24:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:24:19.697883 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7954d33f-1ab2-43ec-b765-27a3db6dc70d None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:24:19.730036 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd1a351-403b-44dd-9947-1f454a30f340 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:24:19.731461 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd1a351-403b-44dd-9947-1f454a30f340 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:24:19.731853 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd1a351-403b-44dd-9947-1f454a30f340 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:24:19.732128 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd1a351-403b-44dd-9947-1f454a30f340 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:24:19.742599 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cd1a351-403b-44dd-9947-1f454a30f340 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 05:24:19.750852 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application [None req-1cd1a351-403b-44dd-9947-1f454a30f340 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 25 05:24:19.750852 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:24:19.750852 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:24:19.750852 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:24:19.750852 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:24:19.750852 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:24:19.750852 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:24:19.750852 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:24:19.750852 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:24:19.750852 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:24:19.750852 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:24:19.750852 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:24:19.750852 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 05:24:19.750852 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 05:24:19.750852 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 05:24:19.750852 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 05:24:19.751852 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 05:24:19.751852 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 05:24:19.751852 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 05:24:19.751852 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 05:24:19.751852 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:24:19.751852 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:24:19.751852 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:24:19.751852 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:24:19.751852 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:24:19.751852 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:24:19.751852 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:24:19.751852 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application with Lock( Jul 25 05:24:19.751852 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:24:19.751852 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:24:19.751852 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:24:19.751852 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:24:19.757716 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:24:19.757716 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:24:19.757716 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:24:19.757716 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:24:19.757716 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 05:24:19.757716 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 05:24:19.757716 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 05:24:19.757716 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 05:24:19.757716 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 05:24:19.757716 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 05:24:19.757716 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 25 05:24:19.757716 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Jul 25 05:24:19.757716 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 292/579] 10.222.0.33 () {56 vars in 1102 bytes} [Mon Jul 25 05:24:19 2022] GET /identity/v3/roles/service => generated 85 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:24:19.757716 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd1a351-403b-44dd-9947-1f454a30f340 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:24:19.814432 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7954d33f-1ab2-43ec-b765-27a3db6dc70d None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:24:19.817843 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:24:19.852878 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59cfff16-6594-4f58-b835-86b2c59d2fff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:24:19.853913 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59cfff16-6594-4f58-b835-86b2c59d2fff None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:24:19.854355 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59cfff16-6594-4f58-b835-86b2c59d2fff None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:24:19.854632 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59cfff16-6594-4f58-b835-86b2c59d2fff None admin] PATH_INFO: `/v3/users/nova` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:24:19.863404 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59cfff16-6594-4f58-b835-86b2c59d2fff None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:24:19.865650 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59cfff16-6594-4f58-b835-86b2c59d2fff None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:24:19.870045 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd1a351-403b-44dd-9947-1f454a30f340 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:24:19.882640 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application [None req-59cfff16-6594-4f58-b835-86b2c59d2fff None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 25 05:24:19.882640 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:24:19.882640 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:24:19.882640 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:24:19.882640 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:24:19.882640 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:24:19.882640 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:24:19.882640 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:24:19.882640 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:24:19.882640 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:24:19.882640 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:24:19.882640 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:24:19.882640 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 25 05:24:19.882640 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 25 05:24:19.882640 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 25 05:24:19.882640 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 25 05:24:19.883604 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:24:19.883604 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:24:19.883604 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 25 05:24:19.883604 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 05:24:19.883604 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 25 05:24:19.883604 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 05:24:19.883604 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:24:19.883604 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:24:19.883604 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:24:19.883604 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:24:19.883604 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:24:19.883604 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application with Lock( Jul 25 05:24:19.883604 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:24:19.883604 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:24:19.883604 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:24:19.883604 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:24:19.883604 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:24:19.883604 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:24:19.884584 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:24:19.884584 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:24:19.884584 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 25 05:24:19.884584 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 25 05:24:19.884584 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 25 05:24:19.884584 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 25 05:24:19.884584 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 25 05:24:19.884584 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 25 05:24:19.884584 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 25 05:24:19.884584 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Jul 25 05:24:19.884584 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 288/580] 10.222.0.33 () {56 vars in 1093 bytes} [Mon Jul 25 05:24:19 2022] GET /identity/v3/users/nova => generated 82 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:24:19.884584 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59cfff16-6594-4f58-b835-86b2c59d2fff None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:24:19.910187 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c512c2e4-0072-4652-9e79-60fc75910135 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:24:19.910858 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c512c2e4-0072-4652-9e79-60fc75910135 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:24:19.911147 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c512c2e4-0072-4652-9e79-60fc75910135 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:24:19.911441 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c512c2e4-0072-4652-9e79-60fc75910135 None admin] PATH_INFO: `/v3/domains` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:24:19.912161 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c512c2e4-0072-4652-9e79-60fc75910135 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:24:19.912477 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c512c2e4-0072-4652-9e79-60fc75910135 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:24:19.913894 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c512c2e4-0072-4652-9e79-60fc75910135 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:24:19.926304 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 293/581] 10.222.0.33 () {56 vars in 1109 bytes} [Mon Jul 25 05:24:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:24:19.927118 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c512c2e4-0072-4652-9e79-60fc75910135 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:24:19.946550 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:24:19.995322 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59cfff16-6594-4f58-b835-86b2c59d2fff None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:24:20.030630 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-986d89a0-3922-4baf-8b21-07a5b3473387 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:24:20.031454 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986d89a0-3922-4baf-8b21-07a5b3473387 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:24:20.031744 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986d89a0-3922-4baf-8b21-07a5b3473387 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:24:20.032026 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986d89a0-3922-4baf-8b21-07a5b3473387 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:24:20.037917 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c512c2e4-0072-4652-9e79-60fc75910135 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:24:20.039596 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-986d89a0-3922-4baf-8b21-07a5b3473387 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:24:20.041115 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-986d89a0-3922-4baf-8b21-07a5b3473387 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:24:20.055502 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application [None req-986d89a0-3922-4baf-8b21-07a5b3473387 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 25 05:24:20.055502 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:24:20.055502 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:24:20.055502 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:24:20.055502 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:24:20.055502 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:24:20.055502 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:24:20.055502 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:24:20.055502 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:24:20.055502 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:24:20.055502 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:24:20.055502 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:24:20.055502 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 25 05:24:20.055502 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 25 05:24:20.055502 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 25 05:24:20.055502 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 25 05:24:20.056450 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:24:20.056450 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:24:20.056450 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 25 05:24:20.056450 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 05:24:20.056450 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 25 05:24:20.056450 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 05:24:20.056450 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:24:20.056450 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:24:20.056450 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:24:20.056450 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:24:20.056450 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:24:20.056450 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application with Lock( Jul 25 05:24:20.056450 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:24:20.056450 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:24:20.056450 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:24:20.056450 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:24:20.056450 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:24:20.056450 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:24:20.057468 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:24:20.057468 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:24:20.057468 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 25 05:24:20.057468 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 25 05:24:20.057468 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 25 05:24:20.057468 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 25 05:24:20.057468 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 25 05:24:20.057468 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 25 05:24:20.057468 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 25 05:24:20.057468 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Jul 25 05:24:20.057468 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 289/582] 10.222.0.33 () {56 vars in 1099 bytes} [Mon Jul 25 05:24:19 2022] GET /identity/v3/users/cinder => generated 84 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:24:20.057468 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-986d89a0-3922-4baf-8b21-07a5b3473387 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:24:20.080763 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faef3afc-ec77-4d5a-8230-b6062e99284e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:24:20.081859 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faef3afc-ec77-4d5a-8230-b6062e99284e None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:24:20.082288 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faef3afc-ec77-4d5a-8230-b6062e99284e None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:24:20.082748 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faef3afc-ec77-4d5a-8230-b6062e99284e None admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:24:20.083647 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faef3afc-ec77-4d5a-8230-b6062e99284e None admin] RBAC: Adding query filter params (name=service) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:24:20.084157 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faef3afc-ec77-4d5a-8230-b6062e99284e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:24:20.085680 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faef3afc-ec77-4d5a-8230-b6062e99284e None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:24:20.087224 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:24:20.095899 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 294/583] 10.222.0.33 () {56 vars in 1103 bytes} [Mon Jul 25 05:24:19 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:24:20.096935 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faef3afc-ec77-4d5a-8230-b6062e99284e None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:24:20.170569 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-986d89a0-3922-4baf-8b21-07a5b3473387 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:24:20.181220 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:24:20.204644 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faef3afc-ec77-4d5a-8230-b6062e99284e None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:24:20.209547 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38fc711b-4ad1-4feb-a657-2aea81c065f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:24:20.210409 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38fc711b-4ad1-4feb-a657-2aea81c065f5 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:24:20.210859 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38fc711b-4ad1-4feb-a657-2aea81c065f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:24:20.211301 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38fc711b-4ad1-4feb-a657-2aea81c065f5 None admin] PATH_INFO: `/v3/users` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:24:20.212371 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38fc711b-4ad1-4feb-a657-2aea81c065f5 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:24:20.212861 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38fc711b-4ad1-4feb-a657-2aea81c065f5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:24:20.214611 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38fc711b-4ad1-4feb-a657-2aea81c065f5 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:24:20.234471 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 290/584] 10.222.0.33 () {56 vars in 1133 bytes} [Mon Jul 25 05:24:20 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:24:20.235510 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38fc711b-4ad1-4feb-a657-2aea81c065f5 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:24:20.242674 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-727287b0-70da-478b-90c4-9aa0204392f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:24:20.243474 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727287b0-70da-478b-90c4-9aa0204392f1 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:24:20.243758 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727287b0-70da-478b-90c4-9aa0204392f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:24:20.244031 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727287b0-70da-478b-90c4-9aa0204392f1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:24:20.251858 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-727287b0-70da-478b-90c4-9aa0204392f1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:24:20.253334 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-727287b0-70da-478b-90c4-9aa0204392f1 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:24:20.267697 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application [None req-727287b0-70da-478b-90c4-9aa0204392f1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 05:24:20.267697 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:24:20.267697 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:24:20.267697 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:24:20.267697 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:24:20.267697 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:24:20.267697 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:24:20.267697 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:24:20.267697 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:24:20.267697 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:24:20.267697 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:24:20.267697 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:24:20.267697 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 25 05:24:20.267697 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 25 05:24:20.267697 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 25 05:24:20.268806 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 25 05:24:20.268806 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:24:20.268806 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:24:20.268806 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:24:20.268806 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:24:20.268806 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:24:20.268806 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:24:20.268806 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:24:20.268806 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application with Lock( Jul 25 05:24:20.268806 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:24:20.268806 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:24:20.268806 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:24:20.268806 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:24:20.268806 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:24:20.268806 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:24:20.268806 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:24:20.268806 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:24:20.269790 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 25 05:24:20.269790 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 25 05:24:20.269790 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 25 05:24:20.269790 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 25 05:24:20.269790 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 25 05:24:20.269790 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 25 05:24:20.269790 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 05:24:20.269790 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Jul 25 05:24:20.269790 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 295/585] 10.222.0.33 () {56 vars in 1111 bytes} [Mon Jul 25 05:24:20 2022] GET /identity/v3/projects/service => generated 88 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:24:20.270611 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-727287b0-70da-478b-90c4-9aa0204392f1 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:24:20.327338 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:24:20.349995 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38fc711b-4ad1-4feb-a657-2aea81c065f5 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:24:20.373905 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-727287b0-70da-478b-90c4-9aa0204392f1 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:24:20.390233 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba88928-b6a3-4dd7-b2e6-5d5769c5695e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:24:20.390915 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba88928-b6a3-4dd7-b2e6-5d5769c5695e None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:24:20.391204 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba88928-b6a3-4dd7-b2e6-5d5769c5695e None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:24:20.391480 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba88928-b6a3-4dd7-b2e6-5d5769c5695e None admin] PATH_INFO: `/v3/users` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:24:20.392319 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ba88928-b6a3-4dd7-b2e6-5d5769c5695e None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:24:20.392629 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ba88928-b6a3-4dd7-b2e6-5d5769c5695e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:24:20.394094 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ba88928-b6a3-4dd7-b2e6-5d5769c5695e None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:24:20.408236 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d187c1-6324-4052-ab43-f5961fff8a85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:24:20.409062 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d187c1-6324-4052-ab43-f5961fff8a85 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:24:20.409351 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d187c1-6324-4052-ab43-f5961fff8a85 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:24:20.409631 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d187c1-6324-4052-ab43-f5961fff8a85 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:24:20.412313 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 291/586] 10.222.0.33 () {56 vars in 1137 bytes} [Mon Jul 25 05:24:20 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:24:20.413095 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:24:20.417399 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba88928-b6a3-4dd7-b2e6-5d5769c5695e None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:24:20.422118 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74d187c1-6324-4052-ab43-f5961fff8a85 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:24:20.423584 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74d187c1-6324-4052-ab43-f5961fff8a85 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:24:20.442941 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application [None req-74d187c1-6324-4052-ab43-f5961fff8a85 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 05:24:20.442941 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:24:20.442941 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:24:20.442941 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:24:20.442941 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:24:20.442941 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:24:20.442941 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:24:20.442941 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:24:20.442941 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:24:20.442941 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:24:20.442941 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:24:20.442941 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:24:20.442941 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 05:24:20.442941 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 05:24:20.442941 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 05:24:20.443858 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 05:24:20.443858 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:24:20.443858 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:24:20.443858 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:24:20.443858 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:24:20.443858 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:24:20.443858 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:24:20.443858 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:24:20.443858 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application with Lock( Jul 25 05:24:20.443858 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:24:20.443858 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:24:20.443858 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:24:20.443858 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:24:20.443858 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:24:20.443858 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:24:20.443858 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:24:20.443858 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:24:20.444754 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 05:24:20.444754 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 05:24:20.444754 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 05:24:20.444754 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Jul 25 05:24:20.444754 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 296/587] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Jul 25 05:24:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:24:20.444754 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d187c1-6324-4052-ab43-f5961fff8a85 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:24:20.524030 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba88928-b6a3-4dd7-b2e6-5d5769c5695e None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:24:20.554091 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d187c1-6324-4052-ab43-f5961fff8a85 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:24:20.562747 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-036120ed-9f1f-4b3b-9c03-b1ed5b96f0b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:24:20.564236 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036120ed-9f1f-4b3b-9c03-b1ed5b96f0b6 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:24:20.564525 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036120ed-9f1f-4b3b-9c03-b1ed5b96f0b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:24:20.564831 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036120ed-9f1f-4b3b-9c03-b1ed5b96f0b6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:24:20.573706 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:24:20.578013 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-036120ed-9f1f-4b3b-9c03-b1ed5b96f0b6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:24:20.580667 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-036120ed-9f1f-4b3b-9c03-b1ed5b96f0b6 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:24:20.589964 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f02771d-9981-4645-ab92-bddb1acf3dfa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:24:20.590634 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f02771d-9981-4645-ab92-bddb1acf3dfa None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:24:20.590976 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f02771d-9981-4645-ab92-bddb1acf3dfa None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:24:20.591258 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f02771d-9981-4645-ab92-bddb1acf3dfa None admin] PATH_INFO: `/v3/projects` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:24:20.592048 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f02771d-9981-4645-ab92-bddb1acf3dfa None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:24:20.592354 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f02771d-9981-4645-ab92-bddb1acf3dfa None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:24:20.593807 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f02771d-9981-4645-ab92-bddb1acf3dfa None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:24:20.605270 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application [None req-036120ed-9f1f-4b3b-9c03-b1ed5b96f0b6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 05:24:20.605270 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:24:20.605270 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:24:20.605270 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:24:20.605270 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:24:20.605270 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:24:20.605270 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:24:20.605270 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:24:20.605270 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:24:20.605270 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:24:20.605270 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:24:20.605270 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:24:20.605270 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 05:24:20.605270 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 05:24:20.605270 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 05:24:20.607261 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 05:24:20.607261 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:24:20.607261 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:24:20.607261 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:24:20.607261 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:24:20.607261 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:24:20.607261 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:24:20.607261 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:24:20.607261 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application with Lock( Jul 25 05:24:20.607261 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:24:20.607261 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:24:20.607261 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:24:20.607261 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:24:20.607261 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:24:20.607261 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:24:20.607261 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:24:20.607261 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:24:20.608169 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 05:24:20.608169 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 05:24:20.608169 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 05:24:20.608169 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Jul 25 05:24:20.608493 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 297/588] 10.222.0.33 () {56 vars in 1148 bytes} [Mon Jul 25 05:24:20 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:24:20.608493 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f02771d-9981-4645-ab92-bddb1acf3dfa None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:24:20.615247 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 292/589] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Jul 25 05:24:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:24:20.615247 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-036120ed-9f1f-4b3b-9c03-b1ed5b96f0b6 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:24:20.665081 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:24:20.722186 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f02771d-9981-4645-ab92-bddb1acf3dfa None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:24:20.726965 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-036120ed-9f1f-4b3b-9c03-b1ed5b96f0b6 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:24:20.757638 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7181beb-5d84-4c4c-a6ef-ad0437e8f25d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:24:20.758437 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7181beb-5d84-4c4c-a6ef-ad0437e8f25d None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:24:20.758726 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7181beb-5d84-4c4c-a6ef-ad0437e8f25d None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:24:20.759007 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7181beb-5d84-4c4c-a6ef-ad0437e8f25d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:24:20.763642 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0fc402f-5005-4c52-a338-ea8dd62d85a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:24:20.764301 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0fc402f-5005-4c52-a338-ea8dd62d85a9 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:24:20.764590 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0fc402f-5005-4c52-a338-ea8dd62d85a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:24:20.764892 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0fc402f-5005-4c52-a338-ea8dd62d85a9 None admin] PATH_INFO: `/v3/domains` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:24:20.766009 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0fc402f-5005-4c52-a338-ea8dd62d85a9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:24:20.766327 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0fc402f-5005-4c52-a338-ea8dd62d85a9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:24:20.768107 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0fc402f-5005-4c52-a338-ea8dd62d85a9 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:24:20.769756 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7181beb-5d84-4c4c-a6ef-ad0437e8f25d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:24:20.770644 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7181beb-5d84-4c4c-a6ef-ad0437e8f25d None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:24:20.785429 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 293/590] 10.222.0.33 () {56 vars in 1109 bytes} [Mon Jul 25 05:24:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:24:20.785429 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0fc402f-5005-4c52-a338-ea8dd62d85a9 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:24:20.794894 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application [None req-d7181beb-5d84-4c4c-a6ef-ad0437e8f25d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 05:24:20.794894 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:24:20.794894 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:24:20.794894 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:24:20.794894 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:24:20.794894 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:24:20.794894 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:24:20.794894 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:24:20.794894 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:24:20.794894 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:24:20.794894 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:24:20.794894 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:24:20.794894 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 05:24:20.794894 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 05:24:20.794894 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 05:24:20.795989 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 05:24:20.795989 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:24:20.795989 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:24:20.795989 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:24:20.795989 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:24:20.795989 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:24:20.795989 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:24:20.795989 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:24:20.795989 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application with Lock( Jul 25 05:24:20.795989 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:24:20.795989 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:24:20.795989 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:24:20.795989 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:24:20.795989 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:24:20.795989 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:24:20.795989 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:24:20.795989 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:24:20.797014 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 05:24:20.797014 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 05:24:20.797014 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 05:24:20.797014 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Jul 25 05:24:20.797014 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 298/591] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Jul 25 05:24:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:24:20.797014 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7181beb-5d84-4c4c-a6ef-ad0437e8f25d None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:24:20.850475 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:24:20.901737 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:24:20.906390 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7181beb-5d84-4c4c-a6ef-ad0437e8f25d None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:24:20.907670 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0fc402f-5005-4c52-a338-ea8dd62d85a9 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:24:20.942192 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf43a6ef-e1b1-463d-bea0-4681d7dab226 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:24:20.943083 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf43a6ef-e1b1-463d-bea0-4681d7dab226 None admin] REQUEST_METHOD: `PUT` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:24:20.943366 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf43a6ef-e1b1-463d-bea0-4681d7dab226 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:24:20.943641 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf43a6ef-e1b1-463d-bea0-4681d7dab226 None admin] PATH_INFO: `/v3/projects/92afba266cce4d1bad4134c351e74be3/users/de99e44bdf9c4030adbe9c0efa3831cb/roles/200e95cafbc54269a51756ddd406082f` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:24:20.944383 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3547418-91a9-4aa8-8000-378061d6eb8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:24:20.945406 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3547418-91a9-4aa8-8000-378061d6eb8b None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:24:20.945936 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3547418-91a9-4aa8-8000-378061d6eb8b None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:24:20.946356 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3547418-91a9-4aa8-8000-378061d6eb8b None admin] PATH_INFO: `/v3/domains` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:24:20.947346 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3547418-91a9-4aa8-8000-378061d6eb8b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:24:20.947796 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3547418-91a9-4aa8-8000-378061d6eb8b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:24:20.949355 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3547418-91a9-4aa8-8000-378061d6eb8b None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:24:20.958357 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf43a6ef-e1b1-463d-bea0-4681d7dab226 None admin] RBAC: Authorizing `identity:create_grant(project_id=92afba266cce4d1bad4134c351e74be3, user_id=de99e44bdf9c4030adbe9c0efa3831cb, role_id=200e95cafbc54269a51756ddd406082f)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:24:20.959776 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf43a6ef-e1b1-463d-bea0-4681d7dab226 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:24:20.963026 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 299/592] 10.222.0.33 () {56 vars in 1109 bytes} [Mon Jul 25 05:24:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:24:20.963976 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3547418-91a9-4aa8-8000-378061d6eb8b None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:24:20.978315 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 294/593] 10.222.0.33 () {58 vars in 1439 bytes} [Mon Jul 25 05:24:20 2022] PUT /identity/v3/projects/92afba266cce4d1bad4134c351e74be3/users/de99e44bdf9c4030adbe9c0efa3831cb/roles/200e95cafbc54269a51756ddd406082f => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 05:24:20.979140 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf43a6ef-e1b1-463d-bea0-4681d7dab226 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:24:21.095711 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3547418-91a9-4aa8-8000-378061d6eb8b None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:24:21.101529 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf43a6ef-e1b1-463d-bea0-4681d7dab226 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:24:21.103428 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:24:21.151689 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b052524a-b0b9-4c71-b66b-b27fa39c2e44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:24:21.152657 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b052524a-b0b9-4c71-b66b-b27fa39c2e44 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:24:21.153246 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b052524a-b0b9-4c71-b66b-b27fa39c2e44 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:24:21.153246 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b052524a-b0b9-4c71-b66b-b27fa39c2e44 None admin] PATH_INFO: `/v3/domains` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:24:21.154032 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b052524a-b0b9-4c71-b66b-b27fa39c2e44 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:24:21.154343 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b052524a-b0b9-4c71-b66b-b27fa39c2e44 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:24:21.155155 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a525998-9ec8-401c-b7a2-dc61dba81c65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:24:21.156269 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a525998-9ec8-401c-b7a2-dc61dba81c65 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:24:21.156556 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a525998-9ec8-401c-b7a2-dc61dba81c65 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:24:21.156859 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a525998-9ec8-401c-b7a2-dc61dba81c65 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:24:21.158822 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b052524a-b0b9-4c71-b66b-b27fa39c2e44 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:24:21.165462 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a525998-9ec8-401c-b7a2-dc61dba81c65 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:24:21.166949 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a525998-9ec8-401c-b7a2-dc61dba81c65 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:24:21.176185 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 295/594] 10.222.0.33 () {56 vars in 1109 bytes} [Mon Jul 25 05:24:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:24:21.177697 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b052524a-b0b9-4c71-b66b-b27fa39c2e44 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:24:21.185618 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application [None req-7a525998-9ec8-401c-b7a2-dc61dba81c65 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jul 25 05:24:21.185618 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:24:21.185618 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:24:21.185618 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:24:21.185618 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:24:21.185618 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:24:21.185618 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:24:21.185618 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:24:21.185618 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:24:21.185618 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:24:21.185618 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:24:21.185618 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:24:21.185618 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 25 05:24:21.185618 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 25 05:24:21.185618 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 25 05:24:21.185618 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 25 05:24:21.186570 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:24:21.186570 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:24:21.186570 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 25 05:24:21.186570 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 05:24:21.186570 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 25 05:24:21.186570 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 05:24:21.186570 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:24:21.186570 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:24:21.186570 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:24:21.186570 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:24:21.186570 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:24:21.186570 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application with Lock( Jul 25 05:24:21.186570 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:24:21.186570 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:24:21.186570 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:24:21.186570 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:24:21.186570 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:24:21.186570 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:24:21.187500 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:24:21.187500 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:24:21.187500 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 25 05:24:21.187500 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 25 05:24:21.187500 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 25 05:24:21.187500 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 25 05:24:21.187500 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 25 05:24:21.187500 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 25 05:24:21.187500 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jul 25 05:24:21.187500 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Jul 25 05:24:21.187500 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 300/595] 10.222.0.33 () {56 vars in 1102 bytes} [Mon Jul 25 05:24:20 2022] GET /identity/v3/users/neutron => generated 85 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:24:21.188078 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a525998-9ec8-401c-b7a2-dc61dba81c65 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:24:21.215219 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:24:21.304005 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b052524a-b0b9-4c71-b66b-b27fa39c2e44 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:24:21.312663 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a525998-9ec8-401c-b7a2-dc61dba81c65 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:24:21.341143 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-395bee6b-c98a-4552-bfc9-b3cf71f7ea70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:24:21.341939 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395bee6b-c98a-4552-bfc9-b3cf71f7ea70 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:24:21.342229 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395bee6b-c98a-4552-bfc9-b3cf71f7ea70 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:24:21.342510 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395bee6b-c98a-4552-bfc9-b3cf71f7ea70 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:24:21.349407 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-938fb66d-b9dc-45a3-8bdd-eb632bf357e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:24:21.349728 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-395bee6b-c98a-4552-bfc9-b3cf71f7ea70 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:24:21.350710 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938fb66d-b9dc-45a3-8bdd-eb632bf357e9 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:24:21.351186 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-395bee6b-c98a-4552-bfc9-b3cf71f7ea70 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:24:21.351437 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938fb66d-b9dc-45a3-8bdd-eb632bf357e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:24:21.351962 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938fb66d-b9dc-45a3-8bdd-eb632bf357e9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:24:21.360040 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-938fb66d-b9dc-45a3-8bdd-eb632bf357e9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:24:21.361777 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-938fb66d-b9dc-45a3-8bdd-eb632bf357e9 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:24:21.367002 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application [None req-395bee6b-c98a-4552-bfc9-b3cf71f7ea70 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 05:24:21.367002 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:24:21.367002 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:24:21.367002 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:24:21.367002 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:24:21.367002 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:24:21.367002 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:24:21.367002 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:24:21.367002 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:24:21.367002 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:24:21.367002 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:24:21.367002 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:24:21.367002 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 25 05:24:21.367002 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 25 05:24:21.367002 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 25 05:24:21.367892 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 25 05:24:21.367892 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:24:21.367892 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:24:21.367892 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:24:21.367892 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:24:21.367892 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:24:21.367892 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:24:21.367892 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:24:21.367892 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application with Lock( Jul 25 05:24:21.367892 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:24:21.367892 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:24:21.367892 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:24:21.367892 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:24:21.367892 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:24:21.367892 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:24:21.367892 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:24:21.367892 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:24:21.368820 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 25 05:24:21.368820 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 25 05:24:21.368820 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 25 05:24:21.368820 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 25 05:24:21.368820 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 25 05:24:21.368820 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 25 05:24:21.368820 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 05:24:21.368820 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Jul 25 05:24:21.368820 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 296/596] 10.222.0.33 () {56 vars in 1111 bytes} [Mon Jul 25 05:24:21 2022] GET /identity/v3/projects/service => generated 88 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:24:21.369758 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-395bee6b-c98a-4552-bfc9-b3cf71f7ea70 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:24:21.379991 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application [None req-938fb66d-b9dc-45a3-8bdd-eb632bf357e9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 05:24:21.379991 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:24:21.379991 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:24:21.379991 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:24:21.379991 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:24:21.379991 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:24:21.379991 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:24:21.379991 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:24:21.379991 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:24:21.379991 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:24:21.379991 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:24:21.379991 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:24:21.379991 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 25 05:24:21.379991 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 25 05:24:21.379991 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 25 05:24:21.380943 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 25 05:24:21.380943 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:24:21.380943 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:24:21.380943 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:24:21.380943 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:24:21.380943 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:24:21.380943 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:24:21.380943 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:24:21.380943 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application with Lock( Jul 25 05:24:21.380943 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:24:21.380943 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:24:21.380943 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:24:21.380943 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:24:21.380943 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:24:21.380943 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:24:21.380943 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:24:21.380943 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:24:21.381850 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 25 05:24:21.381850 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 25 05:24:21.381850 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 25 05:24:21.381850 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 25 05:24:21.381850 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 25 05:24:21.381850 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 25 05:24:21.381850 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 05:24:21.381850 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Jul 25 05:24:21.389419 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 301/597] 10.222.0.33 () {56 vars in 1111 bytes} [Mon Jul 25 05:24:21 2022] GET /identity/v3/projects/service => generated 88 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:24:21.390502 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-938fb66d-b9dc-45a3-8bdd-eb632bf357e9 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:24:21.397521 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:24:21.477500 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:24:21.485954 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-395bee6b-c98a-4552-bfc9-b3cf71f7ea70 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:24:21.500245 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-938fb66d-b9dc-45a3-8bdd-eb632bf357e9 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:24:21.524656 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed83746d-8335-4349-9736-e11831a17196 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:24:21.525793 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed83746d-8335-4349-9736-e11831a17196 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:24:21.526154 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed83746d-8335-4349-9736-e11831a17196 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:24:21.526505 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed83746d-8335-4349-9736-e11831a17196 None admin] PATH_INFO: `/v3/users` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:24:21.527820 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed83746d-8335-4349-9736-e11831a17196 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:24:21.528264 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed83746d-8335-4349-9736-e11831a17196 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:24:21.530414 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed83746d-8335-4349-9736-e11831a17196 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:24:21.540177 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20dde019-d51e-43a3-a1e7-7093a7da2f4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:24:21.541276 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20dde019-d51e-43a3-a1e7-7093a7da2f4f None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:24:21.542703 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20dde019-d51e-43a3-a1e7-7093a7da2f4f None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:24:21.542703 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20dde019-d51e-43a3-a1e7-7093a7da2f4f None admin] PATH_INFO: `/v3/projects` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:24:21.543392 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20dde019-d51e-43a3-a1e7-7093a7da2f4f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:24:21.544028 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20dde019-d51e-43a3-a1e7-7093a7da2f4f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:24:21.545736 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20dde019-d51e-43a3-a1e7-7093a7da2f4f None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:24:21.552952 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 297/598] 10.222.0.33 () {56 vars in 1139 bytes} [Mon Jul 25 05:24:21 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:24:21.554397 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed83746d-8335-4349-9736-e11831a17196 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:24:21.560391 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 302/599] 10.222.0.33 () {56 vars in 1148 bytes} [Mon Jul 25 05:24:21 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:24:21.561518 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20dde019-d51e-43a3-a1e7-7093a7da2f4f None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:24:21.649315 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:24:21.664589 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20dde019-d51e-43a3-a1e7-7093a7da2f4f None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:24:21.671972 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed83746d-8335-4349-9736-e11831a17196 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:24:21.700248 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298bfed0-a48d-44d4-8cc7-f1a30e5894e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:24:21.701075 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298bfed0-a48d-44d4-8cc7-f1a30e5894e5 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:24:21.701363 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298bfed0-a48d-44d4-8cc7-f1a30e5894e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:24:21.701815 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298bfed0-a48d-44d4-8cc7-f1a30e5894e5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:24:21.709618 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8587df3a-4adb-428f-9567-eb624a7d16cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:24:21.710357 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8587df3a-4adb-428f-9567-eb624a7d16cd None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:24:21.710767 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:24:21.711048 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8587df3a-4adb-428f-9567-eb624a7d16cd None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:24:21.711335 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8587df3a-4adb-428f-9567-eb624a7d16cd None admin] PATH_INFO: `/v3/projects` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:24:21.712289 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8587df3a-4adb-428f-9567-eb624a7d16cd None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:24:21.712667 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8587df3a-4adb-428f-9567-eb624a7d16cd None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:24:21.713373 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-298bfed0-a48d-44d4-8cc7-f1a30e5894e5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:24:21.714303 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8587df3a-4adb-428f-9567-eb624a7d16cd None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:24:21.715360 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-298bfed0-a48d-44d4-8cc7-f1a30e5894e5 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:24:21.731959 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 298/600] 10.222.0.33 () {56 vars in 1148 bytes} [Mon Jul 25 05:24:21 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:24:21.731959 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8587df3a-4adb-428f-9567-eb624a7d16cd None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:24:21.740937 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application [None req-298bfed0-a48d-44d4-8cc7-f1a30e5894e5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 05:24:21.740937 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:24:21.740937 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:24:21.740937 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:24:21.740937 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:24:21.740937 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:24:21.740937 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:24:21.740937 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:24:21.740937 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:24:21.740937 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:24:21.740937 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:24:21.740937 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:24:21.740937 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 05:24:21.740937 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 05:24:21.740937 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 05:24:21.741865 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 05:24:21.741865 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:24:21.741865 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:24:21.741865 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:24:21.741865 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:24:21.741865 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:24:21.741865 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:24:21.741865 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:24:21.741865 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application with Lock( Jul 25 05:24:21.741865 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:24:21.741865 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:24:21.741865 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:24:21.741865 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:24:21.741865 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:24:21.741865 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:24:21.741865 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:24:21.741865 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:24:21.742751 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 05:24:21.742751 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 05:24:21.742751 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 05:24:21.742751 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Jul 25 05:24:21.742751 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 303/601] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Jul 25 05:24:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:24:21.743787 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b2084a-3451-4f22-87b9-8b32318653b0 None None] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:24:21.743787 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b2084a-3451-4f22-87b9-8b32318653b0 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:24:21.743950 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b2084a-3451-4f22-87b9-8b32318653b0 None None] PATH_INFO: `/` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:24:21.748830 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 304/602] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jul 25 05:24:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:24:21.748830 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b2084a-3451-4f22-87b9-8b32318653b0 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:24:21.845379 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8587df3a-4adb-428f-9567-eb624a7d16cd None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:24:21.863032 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b2084a-3451-4f22-87b9-8b32318653b0 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:24:21.879162 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-459bafc9-1918-42bc-8693-04e25ee8f90e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:24:21.880134 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459bafc9-1918-42bc-8693-04e25ee8f90e None admin] REQUEST_METHOD: `PUT` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:24:21.880428 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459bafc9-1918-42bc-8693-04e25ee8f90e None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:24:21.880710 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459bafc9-1918-42bc-8693-04e25ee8f90e None admin] PATH_INFO: `/v3/projects/92afba266cce4d1bad4134c351e74be3/users/1cd482a61e8a420db45bb532e5dae6e2/roles/200e95cafbc54269a51756ddd406082f` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:24:21.893603 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:24:21.899209 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-459bafc9-1918-42bc-8693-04e25ee8f90e None admin] RBAC: Authorizing `identity:create_grant(project_id=92afba266cce4d1bad4134c351e74be3, user_id=1cd482a61e8a420db45bb532e5dae6e2, role_id=200e95cafbc54269a51756ddd406082f)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:24:21.899385 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d6fc60-7e09-4e84-aa56-d9bb953e76fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:24:21.900224 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d6fc60-7e09-4e84-aa56-d9bb953e76fa None admin] REQUEST_METHOD: `PUT` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:24:21.900508 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d6fc60-7e09-4e84-aa56-d9bb953e76fa None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:24:21.900689 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-459bafc9-1918-42bc-8693-04e25ee8f90e None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:24:21.900890 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d6fc60-7e09-4e84-aa56-d9bb953e76fa None admin] PATH_INFO: `/v3/projects/92afba266cce4d1bad4134c351e74be3/users/db57b478c25643dbaf936f90f6e17d0a/roles/200e95cafbc54269a51756ddd406082f` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:24:21.914027 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 299/603] 10.222.0.33 () {58 vars in 1439 bytes} [Mon Jul 25 05:24:21 2022] PUT /identity/v3/projects/92afba266cce4d1bad4134c351e74be3/users/1cd482a61e8a420db45bb532e5dae6e2/roles/200e95cafbc54269a51756ddd406082f => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 05:24:21.914910 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-459bafc9-1918-42bc-8693-04e25ee8f90e None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:24:21.927834 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9d6fc60-7e09-4e84-aa56-d9bb953e76fa None admin] RBAC: Authorizing `identity:create_grant(project_id=92afba266cce4d1bad4134c351e74be3, user_id=db57b478c25643dbaf936f90f6e17d0a, role_id=200e95cafbc54269a51756ddd406082f)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:24:21.930946 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9d6fc60-7e09-4e84-aa56-d9bb953e76fa None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:24:21.953487 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 305/604] 10.222.0.33 () {58 vars in 1439 bytes} [Mon Jul 25 05:24:21 2022] PUT /identity/v3/projects/92afba266cce4d1bad4134c351e74be3/users/db57b478c25643dbaf936f90f6e17d0a/roles/200e95cafbc54269a51756ddd406082f => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 05:24:21.955198 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2f8dd1-c732-488d-898f-41ee8b1e9c70 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:24:21.955628 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2f8dd1-c732-488d-898f-41ee8b1e9c70 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:24:21.956093 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2f8dd1-c732-488d-898f-41ee8b1e9c70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:24:21.979870 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-3a2f8dd1-c732-488d-898f-41ee8b1e9c70 None None] MFA Rules not processed for user `1eae540242d94c10b0ad33960a855f08`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:24:21.982090 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-3a2f8dd1-c732-488d-898f-41ee8b1e9c70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:24:22.031944 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:24:22.081838 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 306/605] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jul 25 05:24:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 05:24:22.107330 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a2f8dd1-c732-488d-898f-41ee8b1e9c70 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:24:22.119469 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-459bafc9-1918-42bc-8693-04e25ee8f90e None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:24:22.196281 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8351bee-9c59-47b0-8b87-00d32c66a943 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:24:22.198898 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8351bee-9c59-47b0-8b87-00d32c66a943 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:24:22.199226 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8351bee-9c59-47b0-8b87-00d32c66a943 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:24:22.199508 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8351bee-9c59-47b0-8b87-00d32c66a943 None admin] PATH_INFO: `/v3/domains` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:24:22.200258 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8351bee-9c59-47b0-8b87-00d32c66a943 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:24:22.200570 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8351bee-9c59-47b0-8b87-00d32c66a943 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:24:22.202246 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8351bee-9c59-47b0-8b87-00d32c66a943 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:24:22.220414 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 300/606] 10.222.0.33 () {56 vars in 1109 bytes} [Mon Jul 25 05:24:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 306 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:24:22.225743 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8351bee-9c59-47b0-8b87-00d32c66a943 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:24:22.279749 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a2f8dd1-c732-488d-898f-41ee8b1e9c70 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:24:22.295539 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:24:22.330147 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c02f842-6f55-4401-99ae-f928897acc26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:24:22.331214 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c02f842-6f55-4401-99ae-f928897acc26 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:24:22.331502 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c02f842-6f55-4401-99ae-f928897acc26 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:24:22.331780 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c02f842-6f55-4401-99ae-f928897acc26 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:24:22.343474 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c02f842-6f55-4401-99ae-f928897acc26 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 05:24:22.355583 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application [None req-5c02f842-6f55-4401-99ae-f928897acc26 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 25 05:24:22.355583 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:24:22.355583 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:24:22.355583 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:24:22.355583 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:24:22.355583 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:24:22.355583 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:24:22.355583 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:24:22.355583 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:24:22.355583 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:24:22.355583 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:24:22.355583 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:24:22.355583 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 05:24:22.355583 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 05:24:22.355583 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 05:24:22.355583 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 05:24:22.356562 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 05:24:22.356562 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 05:24:22.356562 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 05:24:22.356562 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 05:24:22.356562 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:24:22.356562 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:24:22.356562 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:24:22.356562 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:24:22.356562 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:24:22.356562 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:24:22.356562 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:24:22.356562 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application with Lock( Jul 25 05:24:22.356562 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:24:22.356562 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:24:22.356562 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:24:22.356562 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:24:22.357467 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:24:22.357467 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:24:22.357467 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:24:22.357467 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:24:22.357467 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 05:24:22.357467 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 05:24:22.357467 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 05:24:22.357467 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 05:24:22.357467 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 05:24:22.357467 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 05:24:22.357467 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 25 05:24:22.357467 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Jul 25 05:24:22.357467 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 307/607] 10.222.0.33 () {56 vars in 1102 bytes} [Mon Jul 25 05:24:22 2022] GET /identity/v3/roles/service => generated 85 bytes in 250 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 25 05:24:22.366254 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c02f842-6f55-4401-99ae-f928897acc26 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:24:22.413144 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8351bee-9c59-47b0-8b87-00d32c66a943 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:24:22.426714 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:24:22.460257 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6fe51db-faa6-48fb-b165-9289a8af8bff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:24:22.461438 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6fe51db-faa6-48fb-b165-9289a8af8bff None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:24:22.462076 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6fe51db-faa6-48fb-b165-9289a8af8bff None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:24:22.462557 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6fe51db-faa6-48fb-b165-9289a8af8bff None admin] PATH_INFO: `/v3/projects/service` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:24:22.471404 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6fe51db-faa6-48fb-b165-9289a8af8bff None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:24:22.473298 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6fe51db-faa6-48fb-b165-9289a8af8bff None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:24:22.489418 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application [None req-b6fe51db-faa6-48fb-b165-9289a8af8bff None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 05:24:22.489418 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:24:22.489418 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:24:22.489418 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:24:22.489418 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:24:22.489418 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:24:22.489418 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:24:22.489418 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:24:22.489418 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:24:22.489418 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:24:22.489418 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:24:22.489418 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:24:22.489418 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 25 05:24:22.489418 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 25 05:24:22.489418 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 25 05:24:22.490343 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 25 05:24:22.490343 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:24:22.490343 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:24:22.490343 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:24:22.490343 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:24:22.490343 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:24:22.490343 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:24:22.490343 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:24:22.490343 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application with Lock( Jul 25 05:24:22.490343 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:24:22.490343 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:24:22.490343 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:24:22.490343 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:24:22.490343 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:24:22.490343 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:24:22.490343 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:24:22.490343 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:24:22.491259 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 25 05:24:22.491259 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 25 05:24:22.491259 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 25 05:24:22.491259 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 25 05:24:22.491259 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 25 05:24:22.491259 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 25 05:24:22.491259 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 05:24:22.491259 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Jul 25 05:24:22.493947 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 301/608] 10.222.0.33 () {56 vars in 1111 bytes} [Mon Jul 25 05:24:22 2022] GET /identity/v3/projects/service => generated 88 bytes in 269 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 25 05:24:22.500319 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6fe51db-faa6-48fb-b165-9289a8af8bff None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:24:22.561448 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c02f842-6f55-4401-99ae-f928897acc26 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:24:22.600692 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e90c1f5d-9db1-456a-9a3a-475f32587cd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:24:22.601874 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90c1f5d-9db1-456a-9a3a-475f32587cd1 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:24:22.602683 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90c1f5d-9db1-456a-9a3a-475f32587cd1 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:24:22.603168 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90c1f5d-9db1-456a-9a3a-475f32587cd1 None admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:24:22.604148 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e90c1f5d-9db1-456a-9a3a-475f32587cd1 None admin] RBAC: Adding query filter params (name=service) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:24:22.604684 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e90c1f5d-9db1-456a-9a3a-475f32587cd1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:24:22.607288 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e90c1f5d-9db1-456a-9a3a-475f32587cd1 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:24:22.616803 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 308/609] 10.222.0.33 () {56 vars in 1103 bytes} [Mon Jul 25 05:24:22 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:24:22.623256 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e90c1f5d-9db1-456a-9a3a-475f32587cd1 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:24:22.677266 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6fe51db-faa6-48fb-b165-9289a8af8bff None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:24:22.699734 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:24:22.767314 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c8cd50-87b2-4aac-b266-b9ac146cc7e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:24:22.767971 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c8cd50-87b2-4aac-b266-b9ac146cc7e2 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:24:22.768166 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:24:22.768272 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c8cd50-87b2-4aac-b266-b9ac146cc7e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:24:22.768538 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c8cd50-87b2-4aac-b266-b9ac146cc7e2 None admin] PATH_INFO: `/v3/projects` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:24:22.769382 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c8cd50-87b2-4aac-b266-b9ac146cc7e2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:24:22.769695 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c8cd50-87b2-4aac-b266-b9ac146cc7e2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:24:22.771143 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c8cd50-87b2-4aac-b266-b9ac146cc7e2 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:24:22.773635 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e90c1f5d-9db1-456a-9a3a-475f32587cd1 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:24:22.803719 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 302/610] 10.222.0.33 () {56 vars in 1148 bytes} [Mon Jul 25 05:24:22 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 301 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 05:24:22.810113 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c8cd50-87b2-4aac-b266-b9ac146cc7e2 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:24:22.827823 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a21eec2-9bfb-45d3-a8db-ebe53dff591e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:24:22.828628 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a21eec2-9bfb-45d3-a8db-ebe53dff591e None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:24:22.828949 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a21eec2-9bfb-45d3-a8db-ebe53dff591e None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:24:22.829233 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a21eec2-9bfb-45d3-a8db-ebe53dff591e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:24:22.840827 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a21eec2-9bfb-45d3-a8db-ebe53dff591e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:24:22.842659 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a21eec2-9bfb-45d3-a8db-ebe53dff591e None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:24:22.861582 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application [None req-9a21eec2-9bfb-45d3-a8db-ebe53dff591e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 05:24:22.861582 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:24:22.861582 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:24:22.861582 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:24:22.861582 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:24:22.861582 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:24:22.861582 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:24:22.861582 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:24:22.861582 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:24:22.861582 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:24:22.861582 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:24:22.861582 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:24:22.861582 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 05:24:22.861582 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 05:24:22.861582 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 05:24:22.862539 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 05:24:22.862539 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:24:22.862539 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:24:22.862539 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:24:22.862539 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:24:22.862539 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:24:22.862539 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:24:22.862539 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:24:22.862539 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application with Lock( Jul 25 05:24:22.862539 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:24:22.862539 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:24:22.862539 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:24:22.862539 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:24:22.862539 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:24:22.862539 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:24:22.862539 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:24:22.862539 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:24:22.863456 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 05:24:22.863456 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 05:24:22.863456 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 05:24:22.863456 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Jul 25 05:24:22.863456 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 309/611] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Jul 25 05:24:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 25 05:24:22.870771 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a21eec2-9bfb-45d3-a8db-ebe53dff591e None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:24:22.983473 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a21eec2-9bfb-45d3-a8db-ebe53dff591e None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:24:23.022183 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c8cd50-87b2-4aac-b266-b9ac146cc7e2 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:24:23.032661 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e1b083-c31f-4e2a-aa0d-feb11bdd4e5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:24:23.033462 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e1b083-c31f-4e2a-aa0d-feb11bdd4e5c None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:24:23.033757 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e1b083-c31f-4e2a-aa0d-feb11bdd4e5c None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:24:23.034041 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e1b083-c31f-4e2a-aa0d-feb11bdd4e5c None admin] PATH_INFO: `/v3/domains` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:24:23.034786 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6e1b083-c31f-4e2a-aa0d-feb11bdd4e5c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:24:23.035152 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6e1b083-c31f-4e2a-aa0d-feb11bdd4e5c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:24:23.036810 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6e1b083-c31f-4e2a-aa0d-feb11bdd4e5c None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:24:23.056466 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-838aa984-670f-4062-a680-5b8347728638 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:24:23.057402 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838aa984-670f-4062-a680-5b8347728638 None admin] REQUEST_METHOD: `PUT` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:24:23.057684 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838aa984-670f-4062-a680-5b8347728638 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:24:23.057958 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838aa984-670f-4062-a680-5b8347728638 None admin] PATH_INFO: `/v3/projects/92afba266cce4d1bad4134c351e74be3/users/b2931200f99e43d79a50a9f6e02738e6/roles/200e95cafbc54269a51756ddd406082f` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:24:23.069836 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 310/612] 10.222.0.33 () {56 vars in 1109 bytes} [Mon Jul 25 05:24:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:24:23.074920 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-838aa984-670f-4062-a680-5b8347728638 None admin] RBAC: Authorizing `identity:create_grant(project_id=92afba266cce4d1bad4134c351e74be3, user_id=b2931200f99e43d79a50a9f6e02738e6, role_id=200e95cafbc54269a51756ddd406082f)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:24:23.076044 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e1b083-c31f-4e2a-aa0d-feb11bdd4e5c None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:24:23.076251 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-838aa984-670f-4062-a680-5b8347728638 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:24:23.088937 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:24:23.092396 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 303/613] 10.222.0.33 () {58 vars in 1439 bytes} [Mon Jul 25 05:24:22 2022] PUT /identity/v3/projects/92afba266cce4d1bad4134c351e74be3/users/b2931200f99e43d79a50a9f6e02738e6/roles/200e95cafbc54269a51756ddd406082f => generated 0 bytes in 282 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 25 05:24:23.222541 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e1b083-c31f-4e2a-aa0d-feb11bdd4e5c None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:24:23.276052 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6592b82c-7c69-4964-96bf-4db52a37d514 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:24:23.277120 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6592b82c-7c69-4964-96bf-4db52a37d514 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:24:23.277442 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6592b82c-7c69-4964-96bf-4db52a37d514 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:24:23.277721 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6592b82c-7c69-4964-96bf-4db52a37d514 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:24:23.285773 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6592b82c-7c69-4964-96bf-4db52a37d514 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:24:23.287231 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6592b82c-7c69-4964-96bf-4db52a37d514 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:24:23.303019 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application [None req-6592b82c-7c69-4964-96bf-4db52a37d514 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 25 05:24:23.303019 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:24:23.303019 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:24:23.303019 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:24:23.303019 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:24:23.303019 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:24:23.303019 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:24:23.303019 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:24:23.303019 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:24:23.303019 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:24:23.303019 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:24:23.303019 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:24:23.303019 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 25 05:24:23.303019 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 25 05:24:23.303019 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 25 05:24:23.303019 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 25 05:24:23.304035 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:24:23.304035 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:24:23.304035 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 25 05:24:23.304035 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 05:24:23.304035 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 25 05:24:23.304035 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 05:24:23.304035 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:24:23.304035 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:24:23.304035 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:24:23.304035 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:24:23.304035 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:24:23.304035 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application with Lock( Jul 25 05:24:23.304035 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:24:23.304035 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:24:23.304035 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:24:23.304035 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:24:23.304035 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:24:23.304035 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:24:23.305028 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:24:23.305028 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:24:23.305028 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 25 05:24:23.305028 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 25 05:24:23.305028 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 25 05:24:23.305028 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 25 05:24:23.305028 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 25 05:24:23.305028 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 25 05:24:23.305028 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 25 05:24:23.305028 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Jul 25 05:24:23.305028 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 311/614] 10.222.0.33 () {56 vars in 1096 bytes} [Mon Jul 25 05:24:23 2022] GET /identity/v3/users/swift => generated 83 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 25 05:24:23.310897 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-838aa984-670f-4062-a680-5b8347728638 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:24:23.315610 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326a977c-8de0-4a49-9f4c-45da80cd2372 None None] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:24:23.315976 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326a977c-8de0-4a49-9f4c-45da80cd2372 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:24:23.316347 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326a977c-8de0-4a49-9f4c-45da80cd2372 None None] PATH_INFO: `/` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:24:23.317993 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 312/615] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jul 25 05:24:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 25 05:24:23.322713 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42efa83-21df-4599-9a12-73070b91962f None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:24:23.323005 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42efa83-21df-4599-9a12-73070b91962f None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:24:23.323294 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42efa83-21df-4599-9a12-73070b91962f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:24:23.340694 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-d42efa83-21df-4599-9a12-73070b91962f None None] MFA Rules not processed for user `1eae540242d94c10b0ad33960a855f08`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:24:23.341956 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-d42efa83-21df-4599-9a12-73070b91962f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:24:23.342371 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:24:23.403368 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:24:23.411342 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 313/616] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jul 25 05:24:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 25 05:24:23.435674 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d42efa83-21df-4599-9a12-73070b91962f None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:24:23.437953 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-838aa984-670f-4062-a680-5b8347728638 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:24:23.497608 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c4dcf3-2d9f-4562-a93c-2453b7411e59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:24:23.498829 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c4dcf3-2d9f-4562-a93c-2453b7411e59 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:24:23.499325 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c4dcf3-2d9f-4562-a93c-2453b7411e59 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:24:23.499810 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c4dcf3-2d9f-4562-a93c-2453b7411e59 None admin] PATH_INFO: `/v3/users` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:24:23.500910 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65c4dcf3-2d9f-4562-a93c-2453b7411e59 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:24:23.501431 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65c4dcf3-2d9f-4562-a93c-2453b7411e59 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:24:23.503299 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65c4dcf3-2d9f-4562-a93c-2453b7411e59 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:24:23.532749 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 304/617] 10.222.0.33 () {56 vars in 1135 bytes} [Mon Jul 25 05:24:23 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:24:23.538628 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c4dcf3-2d9f-4562-a93c-2453b7411e59 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:24:23.540000 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d42efa83-21df-4599-9a12-73070b91962f None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:24:23.576758 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da76416-0c0b-465a-9fa0-addedbd7815f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:24:23.577622 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da76416-0c0b-465a-9fa0-addedbd7815f None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:24:23.577917 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da76416-0c0b-465a-9fa0-addedbd7815f None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:24:23.578200 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da76416-0c0b-465a-9fa0-addedbd7815f None admin] PATH_INFO: `/v3/roles/service` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:24:23.589093 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3da76416-0c0b-465a-9fa0-addedbd7815f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 05:24:23.598005 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application [None req-3da76416-0c0b-465a-9fa0-addedbd7815f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 25 05:24:23.598005 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:24:23.598005 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:24:23.598005 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:24:23.598005 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:24:23.598005 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:24:23.598005 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:24:23.598005 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:24:23.598005 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:24:23.598005 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:24:23.598005 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:24:23.598005 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:24:23.598005 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 05:24:23.598005 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 05:24:23.598005 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 05:24:23.598005 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 05:24:23.599015 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 05:24:23.599015 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 05:24:23.599015 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 05:24:23.599015 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 05:24:23.599015 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:24:23.599015 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:24:23.599015 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:24:23.599015 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:24:23.599015 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:24:23.599015 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:24:23.599015 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:24:23.599015 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application with Lock( Jul 25 05:24:23.599015 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:24:23.599015 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:24:23.599015 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:24:23.599015 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:24:23.600338 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:24:23.600338 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:24:23.600338 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:24:23.600338 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:24:23.600338 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 05:24:23.600338 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 05:24:23.600338 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 05:24:23.600338 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 05:24:23.600338 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 05:24:23.600338 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 05:24:23.600338 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 25 05:24:23.600338 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Jul 25 05:24:23.600338 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 314/618] 10.222.0.33 () {56 vars in 1102 bytes} [Mon Jul 25 05:24:23 2022] GET /identity/v3/roles/service => generated 85 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 25 05:24:23.606463 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da76416-0c0b-465a-9fa0-addedbd7815f None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:24:23.669475 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:24:23.712873 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da76416-0c0b-465a-9fa0-addedbd7815f None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:24:23.721321 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:24:23.752274 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f415261-dab4-437a-b81b-22affd4e96bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:24:23.752985 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f415261-dab4-437a-b81b-22affd4e96bf None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:24:23.753271 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f415261-dab4-437a-b81b-22affd4e96bf None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:24:23.753550 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f415261-dab4-437a-b81b-22affd4e96bf None admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:24:23.754314 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f415261-dab4-437a-b81b-22affd4e96bf None admin] RBAC: Adding query filter params (name=service) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:24:23.754625 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f415261-dab4-437a-b81b-22affd4e96bf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:24:23.756006 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f415261-dab4-437a-b81b-22affd4e96bf None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:24:23.771448 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 315/619] 10.222.0.33 () {56 vars in 1103 bytes} [Mon Jul 25 05:24:23 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 05:24:23.776906 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f415261-dab4-437a-b81b-22affd4e96bf None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:24:23.778806 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c4dcf3-2d9f-4562-a93c-2453b7411e59 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:24:23.853491 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c64e83ed-7a3a-454a-9f96-9c247fdbb612 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:24:23.854304 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64e83ed-7a3a-454a-9f96-9c247fdbb612 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:24:23.854591 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64e83ed-7a3a-454a-9f96-9c247fdbb612 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:24:23.854869 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64e83ed-7a3a-454a-9f96-9c247fdbb612 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:24:23.871404 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c64e83ed-7a3a-454a-9f96-9c247fdbb612 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:24:23.880940 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c64e83ed-7a3a-454a-9f96-9c247fdbb612 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:24:23.886522 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f415261-dab4-437a-b81b-22affd4e96bf None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:24:23.908255 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application [None req-c64e83ed-7a3a-454a-9f96-9c247fdbb612 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 05:24:23.908255 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:24:23.908255 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:24:23.908255 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:24:23.908255 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:24:23.908255 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:24:23.908255 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:24:23.908255 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:24:23.908255 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:24:23.908255 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:24:23.908255 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:24:23.908255 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:24:23.908255 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 05:24:23.908255 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 05:24:23.908255 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 05:24:23.909277 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 05:24:23.909277 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:24:23.909277 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:24:23.909277 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:24:23.909277 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:24:23.909277 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:24:23.909277 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:24:23.909277 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:24:23.909277 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application with Lock( Jul 25 05:24:23.909277 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:24:23.909277 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:24:23.909277 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:24:23.909277 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:24:23.909277 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:24:23.909277 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:24:23.909277 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:24:23.909277 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:24:23.910190 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 05:24:23.910190 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 05:24:23.910190 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 05:24:23.910190 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Jul 25 05:24:23.910481 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:24:23.913925 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 305/620] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Jul 25 05:24:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 376 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 25 05:24:23.920107 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c64e83ed-7a3a-454a-9f96-9c247fdbb612 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:24:23.925892 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29341801-c74c-42d4-940e-6c85fc80005f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:24:23.926757 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29341801-c74c-42d4-940e-6c85fc80005f None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:24:23.927053 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29341801-c74c-42d4-940e-6c85fc80005f None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:24:23.927338 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29341801-c74c-42d4-940e-6c85fc80005f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:24:23.939503 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29341801-c74c-42d4-940e-6c85fc80005f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:24:23.941000 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29341801-c74c-42d4-940e-6c85fc80005f None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:24:23.958488 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application [None req-29341801-c74c-42d4-940e-6c85fc80005f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 05:24:23.958488 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:24:23.958488 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:24:23.958488 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:24:23.958488 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:24:23.958488 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:24:23.958488 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:24:23.958488 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:24:23.958488 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:24:23.958488 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:24:23.958488 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:24:23.958488 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:24:23.958488 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 05:24:23.958488 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 05:24:23.958488 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 05:24:23.959418 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 05:24:23.959418 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:24:23.959418 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:24:23.959418 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:24:23.959418 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:24:23.959418 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:24:23.959418 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:24:23.959418 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:24:23.959418 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application with Lock( Jul 25 05:24:23.959418 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:24:23.959418 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:24:23.959418 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:24:23.959418 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:24:23.959418 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:24:23.959418 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:24:23.959418 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:24:23.959418 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:24:23.961449 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 05:24:23.961449 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 05:24:23.961449 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 05:24:23.961449 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Jul 25 05:24:23.961449 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 316/621] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Jul 25 05:24:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:24:23.966150 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29341801-c74c-42d4-940e-6c85fc80005f None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:24:24.069592 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29341801-c74c-42d4-940e-6c85fc80005f None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:24:24.104647 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39334fc7-76db-4dfd-8a53-bbe3e850bd21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:24:24.105430 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39334fc7-76db-4dfd-8a53-bbe3e850bd21 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:24:24.105732 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39334fc7-76db-4dfd-8a53-bbe3e850bd21 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:24:24.106018 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39334fc7-76db-4dfd-8a53-bbe3e850bd21 None admin] PATH_INFO: `/v3/domains` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:24:24.106766 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39334fc7-76db-4dfd-8a53-bbe3e850bd21 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:24:24.107076 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39334fc7-76db-4dfd-8a53-bbe3e850bd21 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:24:24.108467 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39334fc7-76db-4dfd-8a53-bbe3e850bd21 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:24:24.121352 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 317/622] 10.222.0.33 () {56 vars in 1109 bytes} [Mon Jul 25 05:24:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:24:24.123608 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c64e83ed-7a3a-454a-9f96-9c247fdbb612 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:24:24.126913 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39334fc7-76db-4dfd-8a53-bbe3e850bd21 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:24:24.166591 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:24:24.173192 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:24:24.175345 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-726cd8a1-58e6-4d54-8fbe-82605f496e01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:24:24.176274 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726cd8a1-58e6-4d54-8fbe-82605f496e01 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:24:24.176768 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726cd8a1-58e6-4d54-8fbe-82605f496e01 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:24:24.177258 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726cd8a1-58e6-4d54-8fbe-82605f496e01 None admin] PATH_INFO: `/v3/domains` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:24:24.178179 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-726cd8a1-58e6-4d54-8fbe-82605f496e01 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:24:24.178715 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-726cd8a1-58e6-4d54-8fbe-82605f496e01 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:24:24.180300 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-726cd8a1-58e6-4d54-8fbe-82605f496e01 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:24:24.200763 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 306/623] 10.222.0.33 () {56 vars in 1109 bytes} [Mon Jul 25 05:24:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 05:24:24.202493 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-726cd8a1-58e6-4d54-8fbe-82605f496e01 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:24:24.232051 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39334fc7-76db-4dfd-8a53-bbe3e850bd21 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:24:24.289539 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a5ce8d-a08f-4aa2-a9c5-22ea4786aba8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:24:24.290390 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a5ce8d-a08f-4aa2-a9c5-22ea4786aba8 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:24:24.290679 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a5ce8d-a08f-4aa2-a9c5-22ea4786aba8 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:24:24.290958 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a5ce8d-a08f-4aa2-a9c5-22ea4786aba8 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:24:24.299548 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2a5ce8d-a08f-4aa2-a9c5-22ea4786aba8 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:24:24.301034 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2a5ce8d-a08f-4aa2-a9c5-22ea4786aba8 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:24:24.324745 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application [None req-f2a5ce8d-a08f-4aa2-a9c5-22ea4786aba8 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jul 25 05:24:24.324745 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:24:24.324745 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:24:24.324745 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:24:24.324745 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:24:24.324745 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:24:24.324745 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:24:24.324745 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:24:24.324745 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:24:24.324745 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:24:24.324745 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:24:24.324745 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:24:24.324745 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 25 05:24:24.324745 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 25 05:24:24.324745 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 25 05:24:24.324745 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 25 05:24:24.325915 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:24:24.325915 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:24:24.325915 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 25 05:24:24.325915 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 05:24:24.325915 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 25 05:24:24.325915 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 05:24:24.325915 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:24:24.325915 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:24:24.325915 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:24:24.325915 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:24:24.325915 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:24:24.325915 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application with Lock( Jul 25 05:24:24.325915 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:24:24.325915 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:24:24.325915 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:24:24.325915 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:24:24.325915 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:24:24.325915 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:24:24.327304 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:24:24.327304 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:24:24.327304 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 25 05:24:24.327304 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 25 05:24:24.327304 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 25 05:24:24.327304 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 25 05:24:24.327304 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 25 05:24:24.327304 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 25 05:24:24.327304 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jul 25 05:24:24.327304 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Jul 25 05:24:24.327304 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 318/624] 10.222.0.33 () {56 vars in 1099 bytes} [Mon Jul 25 05:24:24 2022] GET /identity/v3/users/glance => generated 84 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 25 05:24:24.327937 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-726cd8a1-58e6-4d54-8fbe-82605f496e01 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:24:24.328024 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b06fc2-dde6-4597-b45f-2990df6b9789 None None] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:24:24.328024 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b06fc2-dde6-4597-b45f-2990df6b9789 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:24:24.328161 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b06fc2-dde6-4597-b45f-2990df6b9789 None None] PATH_INFO: `/` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:24:24.334173 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 319/625] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jul 25 05:24:24 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:24:24.334964 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b06fc2-dde6-4597-b45f-2990df6b9789 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:24:24.371619 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd733cf7-ad36-42fb-b335-cd429736565f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:24:24.372440 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd733cf7-ad36-42fb-b335-cd429736565f None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:24:24.372979 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd733cf7-ad36-42fb-b335-cd429736565f None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:24:24.373269 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd733cf7-ad36-42fb-b335-cd429736565f None admin] PATH_INFO: `/v3/projects/service` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:24:24.381942 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd733cf7-ad36-42fb-b335-cd429736565f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:24:24.383821 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd733cf7-ad36-42fb-b335-cd429736565f None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:24:24.404153 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application [None req-cd733cf7-ad36-42fb-b335-cd429736565f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 05:24:24.404153 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:24:24.404153 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:24:24.404153 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:24:24.404153 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:24:24.404153 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:24:24.404153 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:24:24.404153 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:24:24.404153 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:24:24.404153 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:24:24.404153 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:24:24.404153 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:24:24.404153 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 25 05:24:24.404153 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 25 05:24:24.404153 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 25 05:24:24.405234 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 25 05:24:24.405234 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:24:24.405234 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:24:24.405234 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:24:24.405234 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:24:24.405234 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:24:24.405234 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:24:24.405234 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:24:24.405234 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application with Lock( Jul 25 05:24:24.405234 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:24:24.405234 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:24:24.405234 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:24:24.405234 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:24:24.405234 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:24:24.405234 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:24:24.405234 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:24:24.405234 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:24:24.407237 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 25 05:24:24.407237 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 25 05:24:24.407237 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 25 05:24:24.407237 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 25 05:24:24.407237 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 25 05:24:24.407237 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 25 05:24:24.407237 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 05:24:24.407237 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Jul 25 05:24:24.407237 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 307/626] 10.222.0.33 () {56 vars in 1111 bytes} [Mon Jul 25 05:24:24 2022] GET /identity/v3/projects/service => generated 88 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 25 05:24:24.407237 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ecf81e6-97c5-4453-874f-80eec264a62d None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:24:24.407237 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ecf81e6-97c5-4453-874f-80eec264a62d None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:24:24.408675 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ecf81e6-97c5-4453-874f-80eec264a62d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:24:24.430114 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-0ecf81e6-97c5-4453-874f-80eec264a62d None None] MFA Rules not processed for user `1eae540242d94c10b0ad33960a855f08`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:24:24.431334 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-0ecf81e6-97c5-4453-874f-80eec264a62d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:24:24.443660 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:24:24.455168 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b06fc2-dde6-4597-b45f-2990df6b9789 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:24:24.489170 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:24:24.496663 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d00d2990-e6bc-4edd-a09a-74a0035c629f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:24:24.497599 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00d2990-e6bc-4edd-a09a-74a0035c629f None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:24:24.498073 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00d2990-e6bc-4edd-a09a-74a0035c629f None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:24:24.498562 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00d2990-e6bc-4edd-a09a-74a0035c629f None admin] PATH_INFO: `/v3/users` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:24:24.500133 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d00d2990-e6bc-4edd-a09a-74a0035c629f None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:24:24.500747 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d00d2990-e6bc-4edd-a09a-74a0035c629f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:24:24.502885 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d00d2990-e6bc-4edd-a09a-74a0035c629f None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:24:24.514797 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 308/627] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jul 25 05:24:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 05:24:24.521816 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ecf81e6-97c5-4453-874f-80eec264a62d None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:24:24.523817 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 320/628] 10.222.0.33 () {56 vars in 1137 bytes} [Mon Jul 25 05:24:24 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:24:24.530674 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d00d2990-e6bc-4edd-a09a-74a0035c629f None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:24:24.644139 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ecf81e6-97c5-4453-874f-80eec264a62d None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:24:24.658900 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d00d2990-e6bc-4edd-a09a-74a0035c629f None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:24:24.679954 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e5ebe7-383b-4260-9f52-d28e09765b09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:24:24.680751 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e5ebe7-383b-4260-9f52-d28e09765b09 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:24:24.681121 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e5ebe7-383b-4260-9f52-d28e09765b09 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:24:24.681467 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e5ebe7-383b-4260-9f52-d28e09765b09 None admin] PATH_INFO: `/v3/projects` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:24:24.682347 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20e5ebe7-383b-4260-9f52-d28e09765b09 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:24:24.682689 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20e5ebe7-383b-4260-9f52-d28e09765b09 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:24:24.684824 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20e5ebe7-383b-4260-9f52-d28e09765b09 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:24:24.701957 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 309/629] 10.222.0.33 () {56 vars in 1148 bytes} [Mon Jul 25 05:24:24 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:24:24.702927 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-517dbc3a-e088-48f8-95a8-4e9e5e233e04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:24:24.703402 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e5ebe7-383b-4260-9f52-d28e09765b09 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:24:24.703742 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517dbc3a-e088-48f8-95a8-4e9e5e233e04 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:24:24.704105 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517dbc3a-e088-48f8-95a8-4e9e5e233e04 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:24:24.704458 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517dbc3a-e088-48f8-95a8-4e9e5e233e04 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:24:24.732817 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:24:24.759124 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:24:24.787329 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-517dbc3a-e088-48f8-95a8-4e9e5e233e04 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:24:24.788825 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-517dbc3a-e088-48f8-95a8-4e9e5e233e04 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:24:24.826228 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e5ebe7-383b-4260-9f52-d28e09765b09 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:24:24.863896 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce8a4f3-8cb3-4bf2-9cda-35adaa056bf7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:24:24.864693 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce8a4f3-8cb3-4bf2-9cda-35adaa056bf7 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:24:24.865025 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce8a4f3-8cb3-4bf2-9cda-35adaa056bf7 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:24:24.865306 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce8a4f3-8cb3-4bf2-9cda-35adaa056bf7 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:24:24.876854 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ce8a4f3-8cb3-4bf2-9cda-35adaa056bf7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 05:24:24.885289 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application [None req-9ce8a4f3-8cb3-4bf2-9cda-35adaa056bf7 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 25 05:24:24.885289 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:24:24.885289 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:24:24.885289 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:24:24.885289 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:24:24.885289 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:24:24.885289 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:24:24.885289 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:24:24.885289 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:24:24.885289 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:24:24.885289 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:24:24.885289 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:24:24.885289 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 05:24:24.885289 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 05:24:24.885289 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 05:24:24.885289 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 05:24:24.886263 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 05:24:24.886263 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 05:24:24.886263 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 05:24:24.886263 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 05:24:24.886263 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:24:24.886263 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:24:24.886263 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:24:24.886263 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:24:24.886263 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:24:24.886263 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:24:24.886263 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:24:24.886263 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application with Lock( Jul 25 05:24:24.886263 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:24:24.886263 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:24:24.886263 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:24:24.886263 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:24:24.887696 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:24:24.887696 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:24:24.887696 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:24:24.887696 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:24:24.887696 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 05:24:24.887696 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 05:24:24.887696 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 05:24:24.887696 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 05:24:24.887696 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 05:24:24.887696 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 05:24:24.887696 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 25 05:24:24.887696 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Jul 25 05:24:24.887696 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 310/630] 10.222.0.33 () {56 vars in 1102 bytes} [Mon Jul 25 05:24:24 2022] GET /identity/v3/roles/service => generated 85 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:24:24.889149 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce8a4f3-8cb3-4bf2-9cda-35adaa056bf7 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:24:24.897379 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 321/631] 10.222.0.33 () {58 vars in 1282 bytes} [Mon Jul 25 05:24:24 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 368 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 25 05:24:24.898650 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5089cf2c-a246-47a3-a464-25974bf6cb25 None None] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:24:24.898943 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5089cf2c-a246-47a3-a464-25974bf6cb25 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:24:24.899223 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5089cf2c-a246-47a3-a464-25974bf6cb25 None None] PATH_INFO: `/` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:24:24.900650 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 322/632] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jul 25 05:24:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:24:24.904691 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5089cf2c-a246-47a3-a464-25974bf6cb25 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:24:24.958324 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:24:24.999674 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:24:25.002767 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce8a4f3-8cb3-4bf2-9cda-35adaa056bf7 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:24:25.015450 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5089cf2c-a246-47a3-a464-25974bf6cb25 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:24:25.040770 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a33e44-f7c8-44b1-a4b4-64c4f9249f22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:24:25.041827 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a33e44-f7c8-44b1-a4b4-64c4f9249f22 None admin] REQUEST_METHOD: `PUT` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:24:25.042416 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a33e44-f7c8-44b1-a4b4-64c4f9249f22 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:24:25.043209 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a33e44-f7c8-44b1-a4b4-64c4f9249f22 None admin] PATH_INFO: `/v3/projects/92afba266cce4d1bad4134c351e74be3/users/eb135a8d5d7444cfbfa336115ba2ba44/roles/200e95cafbc54269a51756ddd406082f` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:24:25.050910 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc66ccd-9cea-49aa-98b0-11dac58c24c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:24:25.051604 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc66ccd-9cea-49aa-98b0-11dac58c24c9 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:24:25.051862 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc66ccd-9cea-49aa-98b0-11dac58c24c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:24:25.052136 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc66ccd-9cea-49aa-98b0-11dac58c24c9 None admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:24:25.053329 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cc66ccd-9cea-49aa-98b0-11dac58c24c9 None admin] RBAC: Adding query filter params (name=service) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:24:25.053707 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cc66ccd-9cea-49aa-98b0-11dac58c24c9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:24:25.055031 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cc66ccd-9cea-49aa-98b0-11dac58c24c9 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:24:25.059992 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58a33e44-f7c8-44b1-a4b4-64c4f9249f22 None admin] RBAC: Authorizing `identity:create_grant(project_id=92afba266cce4d1bad4134c351e74be3, user_id=eb135a8d5d7444cfbfa336115ba2ba44, role_id=200e95cafbc54269a51756ddd406082f)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:24:25.061562 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58a33e44-f7c8-44b1-a4b4-64c4f9249f22 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:24:25.064290 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 323/633] 10.222.0.33 () {56 vars in 1103 bytes} [Mon Jul 25 05:24:24 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:24:25.066273 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc66ccd-9cea-49aa-98b0-11dac58c24c9 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:24:25.084813 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 311/634] 10.222.0.33 () {58 vars in 1439 bytes} [Mon Jul 25 05:24:24 2022] PUT /identity/v3/projects/92afba266cce4d1bad4134c351e74be3/users/eb135a8d5d7444cfbfa336115ba2ba44/roles/200e95cafbc54269a51756ddd406082f => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 05:24:25.084813 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb284c42-115a-4629-9816-66f583dd3bed None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:24:25.084813 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb284c42-115a-4629-9816-66f583dd3bed None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:24:25.084813 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb284c42-115a-4629-9816-66f583dd3bed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:24:25.111290 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-eb284c42-115a-4629-9816-66f583dd3bed None None] MFA Rules not processed for user `1eae540242d94c10b0ad33960a855f08`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:24:25.112512 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-eb284c42-115a-4629-9816-66f583dd3bed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:24:25.196584 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc66ccd-9cea-49aa-98b0-11dac58c24c9 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:24:25.213695 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:24:25.220946 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 312/635] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jul 25 05:24:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 05:24:25.222491 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb284c42-115a-4629-9816-66f583dd3bed None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:24:25.258521 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d39cc5-a664-4660-991f-d2f142536a5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:24:25.259319 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d39cc5-a664-4660-991f-d2f142536a5d None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:24:25.259608 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d39cc5-a664-4660-991f-d2f142536a5d None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:24:25.259888 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d39cc5-a664-4660-991f-d2f142536a5d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:24:25.273380 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1d39cc5-a664-4660-991f-d2f142536a5d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:24:25.275177 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1d39cc5-a664-4660-991f-d2f142536a5d None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:24:25.294106 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application [None req-d1d39cc5-a664-4660-991f-d2f142536a5d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 05:24:25.294106 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:24:25.294106 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:24:25.294106 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:24:25.294106 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:24:25.294106 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:24:25.294106 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:24:25.294106 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:24:25.294106 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:24:25.294106 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:24:25.294106 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:24:25.294106 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:24:25.294106 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 05:24:25.294106 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 05:24:25.294106 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 05:24:25.295259 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 05:24:25.295259 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:24:25.295259 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:24:25.295259 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:24:25.295259 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:24:25.295259 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:24:25.295259 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:24:25.295259 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:24:25.295259 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application with Lock( Jul 25 05:24:25.295259 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:24:25.295259 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:24:25.295259 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:24:25.295259 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:24:25.295259 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:24:25.295259 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:24:25.295259 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:24:25.295259 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:24:25.296370 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 05:24:25.296370 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 05:24:25.296370 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 05:24:25.296370 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Jul 25 05:24:25.298266 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 324/636] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Jul 25 05:24:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:24:25.299216 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d39cc5-a664-4660-991f-d2f142536a5d None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:24:25.341353 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:24:25.374448 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb284c42-115a-4629-9816-66f583dd3bed None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:24:25.414748 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d39cc5-a664-4660-991f-d2f142536a5d None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:24:25.421604 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1aeee1b-c6a3-4f4f-9875-aa977189e3d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:24:25.422865 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1aeee1b-c6a3-4f4f-9875-aa977189e3d4 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:24:25.423249 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1aeee1b-c6a3-4f4f-9875-aa977189e3d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:24:25.424184 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1aeee1b-c6a3-4f4f-9875-aa977189e3d4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:24:25.438530 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1aeee1b-c6a3-4f4f-9875-aa977189e3d4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:24:25.440377 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1aeee1b-c6a3-4f4f-9875-aa977189e3d4 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:24:25.461073 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c3d688-f12d-4ee0-9286-9998fd12a296 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:24:25.461932 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c3d688-f12d-4ee0-9286-9998fd12a296 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:24:25.462315 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c3d688-f12d-4ee0-9286-9998fd12a296 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:24:25.462606 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application [None req-d1aeee1b-c6a3-4f4f-9875-aa977189e3d4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 05:24:25.462606 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:24:25.462606 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:24:25.462606 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:24:25.462606 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:24:25.462606 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:24:25.462606 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:24:25.462606 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:24:25.462606 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:24:25.462606 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:24:25.462606 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:24:25.462606 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:24:25.462606 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 05:24:25.462606 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 05:24:25.462606 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 05:24:25.463625 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 05:24:25.463625 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:24:25.463625 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:24:25.463625 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:24:25.463625 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:24:25.463625 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:24:25.463625 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:24:25.463625 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:24:25.463625 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application with Lock( Jul 25 05:24:25.463625 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:24:25.463625 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:24:25.463625 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:24:25.463625 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:24:25.463625 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:24:25.463625 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:24:25.463625 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:24:25.463625 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:24:25.464538 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 05:24:25.464538 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 05:24:25.464538 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 05:24:25.464538 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Jul 25 05:24:25.465322 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c3d688-f12d-4ee0-9286-9998fd12a296 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:24:25.467302 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 313/637] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Jul 25 05:24:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:24:25.468336 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1aeee1b-c6a3-4f4f-9875-aa977189e3d4 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:24:25.482296 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5c3d688-f12d-4ee0-9286-9998fd12a296 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 05:24:25.492708 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application [None req-c5c3d688-f12d-4ee0-9286-9998fd12a296 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 25 05:24:25.492708 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:24:25.492708 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:24:25.492708 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:24:25.492708 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:24:25.492708 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:24:25.492708 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:24:25.492708 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:24:25.492708 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:24:25.492708 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:24:25.492708 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:24:25.492708 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:24:25.492708 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 05:24:25.492708 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 05:24:25.492708 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 05:24:25.492708 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 05:24:25.493687 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 05:24:25.493687 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 05:24:25.493687 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 05:24:25.493687 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 05:24:25.493687 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:24:25.493687 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:24:25.493687 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:24:25.493687 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:24:25.493687 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:24:25.493687 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:24:25.493687 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:24:25.493687 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application with Lock( Jul 25 05:24:25.493687 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:24:25.493687 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:24:25.493687 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:24:25.493687 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:24:25.494612 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:24:25.494612 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:24:25.494612 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:24:25.494612 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:24:25.494612 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 05:24:25.494612 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 05:24:25.494612 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 05:24:25.494612 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 05:24:25.494612 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 05:24:25.494612 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 05:24:25.494612 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 25 05:24:25.494612 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Jul 25 05:24:25.494612 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 325/638] 10.222.0.33 () {56 vars in 1102 bytes} [Mon Jul 25 05:24:25 2022] GET /identity/v3/roles/service => generated 85 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:24:25.495329 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c3d688-f12d-4ee0-9286-9998fd12a296 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:24:25.531315 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:24:25.582723 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1aeee1b-c6a3-4f4f-9875-aa977189e3d4 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:24:25.632560 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c3d688-f12d-4ee0-9286-9998fd12a296 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:24:25.633161 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:24:25.640504 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed17ab6e-345e-4c1c-aae1-68b73fd4726c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:24:25.641210 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed17ab6e-345e-4c1c-aae1-68b73fd4726c None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:24:25.641495 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed17ab6e-345e-4c1c-aae1-68b73fd4726c None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:24:25.641781 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed17ab6e-345e-4c1c-aae1-68b73fd4726c None admin] PATH_INFO: `/v3/domains` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:24:25.643071 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed17ab6e-345e-4c1c-aae1-68b73fd4726c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:24:25.643408 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed17ab6e-345e-4c1c-aae1-68b73fd4726c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:24:25.644810 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed17ab6e-345e-4c1c-aae1-68b73fd4726c None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:24:25.661191 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 314/639] 10.222.0.33 () {56 vars in 1109 bytes} [Mon Jul 25 05:24:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:24:25.662321 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed17ab6e-345e-4c1c-aae1-68b73fd4726c None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:24:25.678112 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa15262-6b55-43f5-9792-d8a6033a986b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:24:25.679077 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa15262-6b55-43f5-9792-d8a6033a986b None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:24:25.679367 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa15262-6b55-43f5-9792-d8a6033a986b None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:24:25.679649 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa15262-6b55-43f5-9792-d8a6033a986b None admin] PATH_INFO: `/v3/domains` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:24:25.680643 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa15262-6b55-43f5-9792-d8a6033a986b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:24:25.680997 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa15262-6b55-43f5-9792-d8a6033a986b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:24:25.683150 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa15262-6b55-43f5-9792-d8a6033a986b None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:24:25.702017 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 326/640] 10.222.0.33 () {56 vars in 1109 bytes} [Mon Jul 25 05:24:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:24:25.703968 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa15262-6b55-43f5-9792-d8a6033a986b None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:24:25.770436 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed17ab6e-345e-4c1c-aae1-68b73fd4726c None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:24:25.792452 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:24:25.815709 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d7412f3-5d6b-4254-b4c1-1f2444de32cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:24:25.816569 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7412f3-5d6b-4254-b4c1-1f2444de32cf None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:24:25.817026 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7412f3-5d6b-4254-b4c1-1f2444de32cf None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:24:25.817298 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa15262-6b55-43f5-9792-d8a6033a986b None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:24:25.817624 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7412f3-5d6b-4254-b4c1-1f2444de32cf None admin] PATH_INFO: `/v3/roles` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:24:25.818514 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d7412f3-5d6b-4254-b4c1-1f2444de32cf None admin] RBAC: Adding query filter params (name=service) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:24:25.818963 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d7412f3-5d6b-4254-b4c1-1f2444de32cf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:24:25.820463 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d7412f3-5d6b-4254-b4c1-1f2444de32cf None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:24:25.829722 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 315/641] 10.222.0.33 () {56 vars in 1103 bytes} [Mon Jul 25 05:24:25 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:24:25.831174 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12fe9325-af00-474c-9de8-a5b679c87a2b None None] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:24:25.831461 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12fe9325-af00-474c-9de8-a5b679c87a2b None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:24:25.831734 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12fe9325-af00-474c-9de8-a5b679c87a2b None None] PATH_INFO: `/` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:24:25.833156 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 316/642] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jul 25 05:24:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:24:25.837573 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12fe9325-af00-474c-9de8-a5b679c87a2b None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:24:25.858707 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c4a7b1-2a4f-4fcc-b708-39741c0fec54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:24:25.859506 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c4a7b1-2a4f-4fcc-b708-39741c0fec54 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:24:25.859790 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c4a7b1-2a4f-4fcc-b708-39741c0fec54 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:24:25.860068 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c4a7b1-2a4f-4fcc-b708-39741c0fec54 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:24:25.866827 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45c4a7b1-2a4f-4fcc-b708-39741c0fec54 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:24:25.868271 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45c4a7b1-2a4f-4fcc-b708-39741c0fec54 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:24:25.882297 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application [None req-45c4a7b1-2a4f-4fcc-b708-39741c0fec54 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 05:24:25.882297 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:24:25.882297 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:24:25.882297 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:24:25.882297 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:24:25.882297 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:24:25.882297 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:24:25.882297 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:24:25.882297 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:24:25.882297 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:24:25.882297 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:24:25.882297 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:24:25.882297 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 25 05:24:25.882297 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 25 05:24:25.882297 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 25 05:24:25.883284 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 25 05:24:25.883284 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:24:25.883284 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:24:25.883284 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:24:25.883284 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:24:25.883284 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:24:25.883284 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:24:25.883284 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:24:25.883284 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application with Lock( Jul 25 05:24:25.883284 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:24:25.883284 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:24:25.883284 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:24:25.883284 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:24:25.883284 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:24:25.883284 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:24:25.883284 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:24:25.883284 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:24:25.884195 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 25 05:24:25.884195 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 25 05:24:25.884195 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 25 05:24:25.884195 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 25 05:24:25.884195 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 25 05:24:25.884195 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 25 05:24:25.884195 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 05:24:25.884195 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Jul 25 05:24:25.888950 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 327/643] 10.222.0.33 () {56 vars in 1111 bytes} [Mon Jul 25 05:24:25 2022] GET /identity/v3/projects/service => generated 88 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:24:25.889653 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c4a7b1-2a4f-4fcc-b708-39741c0fec54 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:24:25.899829 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:24:25.948586 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12fe9325-af00-474c-9de8-a5b679c87a2b None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:24:25.985626 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73dfbc6c-e2bd-4eaa-8832-aa169cd14d67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:24:25.986652 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73dfbc6c-e2bd-4eaa-8832-aa169cd14d67 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:24:25.987086 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73dfbc6c-e2bd-4eaa-8832-aa169cd14d67 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:24:25.987532 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73dfbc6c-e2bd-4eaa-8832-aa169cd14d67 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:24:25.994612 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c4a7b1-2a4f-4fcc-b708-39741c0fec54 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:24:25.995502 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73dfbc6c-e2bd-4eaa-8832-aa169cd14d67 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:24:25.997765 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73dfbc6c-e2bd-4eaa-8832-aa169cd14d67 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:24:26.012286 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application [None req-73dfbc6c-e2bd-4eaa-8832-aa169cd14d67 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 25 05:24:26.012286 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:24:26.012286 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:24:26.012286 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:24:26.012286 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:24:26.012286 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:24:26.012286 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:24:26.012286 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:24:26.012286 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:24:26.012286 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:24:26.012286 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:24:26.012286 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:24:26.012286 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 25 05:24:26.012286 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 25 05:24:26.012286 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 25 05:24:26.012286 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 25 05:24:26.013272 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:24:26.013272 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:24:26.013272 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 25 05:24:26.013272 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 05:24:26.013272 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 25 05:24:26.013272 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 05:24:26.013272 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:24:26.013272 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:24:26.013272 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:24:26.013272 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:24:26.013272 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:24:26.013272 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application with Lock( Jul 25 05:24:26.013272 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:24:26.013272 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:24:26.013272 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:24:26.013272 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:24:26.013272 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:24:26.013272 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:24:26.014304 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:24:26.014304 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:24:26.014304 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 25 05:24:26.014304 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 25 05:24:26.014304 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 25 05:24:26.014304 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 25 05:24:26.014304 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 25 05:24:26.014304 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 25 05:24:26.014304 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 25 05:24:26.014304 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Jul 25 05:24:26.017858 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 317/644] 10.222.0.33 () {56 vars in 1093 bytes} [Mon Jul 25 05:24:25 2022] GET /identity/v3/users/nova => generated 82 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:24:26.019078 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43642546-3ecf-4603-87e1-4d314fe05033 None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:24:26.019379 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43642546-3ecf-4603-87e1-4d314fe05033 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:24:26.019663 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43642546-3ecf-4603-87e1-4d314fe05033 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:24:26.035832 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-43642546-3ecf-4603-87e1-4d314fe05033 None None] MFA Rules not processed for user `1eae540242d94c10b0ad33960a855f08`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:24:26.036465 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-850b407f-e7fc-4baa-aef8-e280ee5bf453 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:24:26.036989 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-43642546-3ecf-4603-87e1-4d314fe05033 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:24:26.037746 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850b407f-e7fc-4baa-aef8-e280ee5bf453 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:24:26.038232 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850b407f-e7fc-4baa-aef8-e280ee5bf453 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:24:26.038708 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850b407f-e7fc-4baa-aef8-e280ee5bf453 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:24:26.051668 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-850b407f-e7fc-4baa-aef8-e280ee5bf453 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:24:26.053335 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-850b407f-e7fc-4baa-aef8-e280ee5bf453 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:24:26.055342 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:24:26.072334 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application [None req-850b407f-e7fc-4baa-aef8-e280ee5bf453 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 05:24:26.072334 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:24:26.072334 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:24:26.072334 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:24:26.072334 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:24:26.072334 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:24:26.072334 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:24:26.072334 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:24:26.072334 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:24:26.072334 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:24:26.072334 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:24:26.072334 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:24:26.072334 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 05:24:26.072334 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 05:24:26.072334 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 05:24:26.073335 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 05:24:26.073335 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:24:26.073335 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:24:26.073335 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:24:26.073335 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:24:26.073335 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:24:26.073335 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:24:26.073335 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:24:26.073335 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application with Lock( Jul 25 05:24:26.073335 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:24:26.073335 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:24:26.073335 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:24:26.073335 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:24:26.073335 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:24:26.073335 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:24:26.073335 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:24:26.073335 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:24:26.074309 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 05:24:26.074309 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 05:24:26.074309 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 05:24:26.074309 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Jul 25 05:24:26.074309 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 328/645] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Jul 25 05:24:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:24:26.074764 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-850b407f-e7fc-4baa-aef8-e280ee5bf453 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:24:26.116849 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 318/646] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jul 25 05:24:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 05:24:26.116849 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43642546-3ecf-4603-87e1-4d314fe05033 None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:24:26.171067 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:24:26.191024 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-850b407f-e7fc-4baa-aef8-e280ee5bf453 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:24:26.226455 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43642546-3ecf-4603-87e1-4d314fe05033 None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:24:26.227706 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b123e8-3c2f-4b2a-bef7-9cc29128ade0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:24:26.228887 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b123e8-3c2f-4b2a-bef7-9cc29128ade0 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:24:26.229174 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b123e8-3c2f-4b2a-bef7-9cc29128ade0 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:24:26.229457 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b123e8-3c2f-4b2a-bef7-9cc29128ade0 None admin] PATH_INFO: `/v3/projects` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:24:26.230305 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0b123e8-3c2f-4b2a-bef7-9cc29128ade0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:24:26.230669 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0b123e8-3c2f-4b2a-bef7-9cc29128ade0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:24:26.232109 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0b123e8-3c2f-4b2a-bef7-9cc29128ade0 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:24:26.245154 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 329/647] 10.222.0.33 () {56 vars in 1148 bytes} [Mon Jul 25 05:24:26 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:24:26.246634 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b123e8-3c2f-4b2a-bef7-9cc29128ade0 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:24:26.268085 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43de6e7e-d162-467d-916c-ffbe7a123aa6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:24:26.268805 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43de6e7e-d162-467d-916c-ffbe7a123aa6 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:24:26.269097 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43de6e7e-d162-467d-916c-ffbe7a123aa6 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:24:26.269375 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43de6e7e-d162-467d-916c-ffbe7a123aa6 None admin] PATH_INFO: `/v3/users` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:24:26.270241 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43de6e7e-d162-467d-916c-ffbe7a123aa6 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:24:26.270555 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43de6e7e-d162-467d-916c-ffbe7a123aa6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:24:26.272048 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43de6e7e-d162-467d-916c-ffbe7a123aa6 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:24:26.290404 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 319/648] 10.222.0.33 () {56 vars in 1133 bytes} [Mon Jul 25 05:24:26 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:24:26.291240 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43de6e7e-d162-467d-916c-ffbe7a123aa6 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:24:26.317677 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:24:26.366318 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b123e8-3c2f-4b2a-bef7-9cc29128ade0 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:24:26.394930 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43de6e7e-d162-467d-916c-ffbe7a123aa6 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:24:26.396823 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:24:26.400227 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c6389b9-1927-4445-a3e9-0c645cf14f25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:24:26.400933 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6389b9-1927-4445-a3e9-0c645cf14f25 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:24:26.401226 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6389b9-1927-4445-a3e9-0c645cf14f25 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:24:26.401507 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6389b9-1927-4445-a3e9-0c645cf14f25 None admin] PATH_INFO: `/v3/domains` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:24:26.402332 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c6389b9-1927-4445-a3e9-0c645cf14f25 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:24:26.402604 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c6389b9-1927-4445-a3e9-0c645cf14f25 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:24:26.403992 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c6389b9-1927-4445-a3e9-0c645cf14f25 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:24:26.415235 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 330/649] 10.222.0.33 () {56 vars in 1109 bytes} [Mon Jul 25 05:24:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:24:26.416044 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c6389b9-1927-4445-a3e9-0c645cf14f25 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:24:26.435158 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c72a83a-8e70-4af0-87e0-a2cdee4877c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:24:26.436407 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c72a83a-8e70-4af0-87e0-a2cdee4877c3 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:24:26.436916 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c72a83a-8e70-4af0-87e0-a2cdee4877c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:24:26.437377 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c72a83a-8e70-4af0-87e0-a2cdee4877c3 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:24:26.449660 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c72a83a-8e70-4af0-87e0-a2cdee4877c3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 05:24:26.457980 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application [None req-8c72a83a-8e70-4af0-87e0-a2cdee4877c3 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 25 05:24:26.457980 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:24:26.457980 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:24:26.457980 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:24:26.457980 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:24:26.457980 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:24:26.457980 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:24:26.457980 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:24:26.457980 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:24:26.457980 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:24:26.457980 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:24:26.457980 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:24:26.457980 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 05:24:26.457980 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 05:24:26.457980 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 05:24:26.457980 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 05:24:26.458919 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 05:24:26.458919 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 05:24:26.458919 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 05:24:26.458919 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 05:24:26.458919 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:24:26.458919 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:24:26.458919 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:24:26.458919 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:24:26.458919 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:24:26.458919 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:24:26.458919 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:24:26.458919 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application with Lock( Jul 25 05:24:26.458919 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:24:26.458919 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:24:26.458919 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:24:26.458919 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:24:26.459944 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:24:26.459944 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:24:26.459944 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:24:26.459944 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:24:26.459944 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 05:24:26.459944 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 05:24:26.459944 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 05:24:26.459944 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 05:24:26.459944 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 05:24:26.459944 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 05:24:26.459944 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 25 05:24:26.459944 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Jul 25 05:24:26.462302 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 320/650] 10.222.0.33 () {56 vars in 1102 bytes} [Mon Jul 25 05:24:26 2022] GET /identity/v3/roles/service => generated 85 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:24:26.463273 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c72a83a-8e70-4af0-87e0-a2cdee4877c3 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:24:26.525492 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c6389b9-1927-4445-a3e9-0c645cf14f25 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:24:26.539487 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:24:26.568049 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5207e64-f7a7-4451-996f-c83d5fb3b2f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:24:26.568917 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5207e64-f7a7-4451-996f-c83d5fb3b2f4 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:24:26.569297 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5207e64-f7a7-4451-996f-c83d5fb3b2f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:24:26.569674 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5207e64-f7a7-4451-996f-c83d5fb3b2f4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:24:26.570741 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5207e64-f7a7-4451-996f-c83d5fb3b2f4 None admin] RBAC: Adding query filter params (role.id=200e95cafbc54269a51756ddd406082f, scope.project.id=92afba266cce4d1bad4134c351e74be3, user.id=de99e44bdf9c4030adbe9c0efa3831cb) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:24:26.571064 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5207e64-f7a7-4451-996f-c83d5fb3b2f4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:24:26.572530 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5207e64-f7a7-4451-996f-c83d5fb3b2f4 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:24:26.578942 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 331/651] 10.222.0.33 () {56 vars in 1374 bytes} [Mon Jul 25 05:24:26 2022] GET /identity/v3/role_assignments?user.id=de99e44bdf9c4030adbe9c0efa3831cb&scope.project.id=92afba266cce4d1bad4134c351e74be3&role.id=200e95cafbc54269a51756ddd406082f => generated 608 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:24:26.579662 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5207e64-f7a7-4451-996f-c83d5fb3b2f4 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:24:26.589924 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c72a83a-8e70-4af0-87e0-a2cdee4877c3 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:24:26.641243 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da7e2c1-0eb0-4cf2-ba35-793c7d1e001f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:24:26.642562 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da7e2c1-0eb0-4cf2-ba35-793c7d1e001f None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:24:26.643062 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da7e2c1-0eb0-4cf2-ba35-793c7d1e001f None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:24:26.643986 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da7e2c1-0eb0-4cf2-ba35-793c7d1e001f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:24:26.682419 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:24:26.717179 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5207e64-f7a7-4451-996f-c83d5fb3b2f4 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:24:26.741535 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0da7e2c1-0eb0-4cf2-ba35-793c7d1e001f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:24:26.744069 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0da7e2c1-0eb0-4cf2-ba35-793c7d1e001f None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:24:26.764602 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f6f0b4-7f5b-4ea0-838f-f0393d27b738 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:24:26.765457 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f6f0b4-7f5b-4ea0-838f-f0393d27b738 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:24:26.765829 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f6f0b4-7f5b-4ea0-838f-f0393d27b738 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:24:26.766009 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f6f0b4-7f5b-4ea0-838f-f0393d27b738 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:24:26.775905 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5f6f0b4-7f5b-4ea0-838f-f0393d27b738 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:24:26.777629 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5f6f0b4-7f5b-4ea0-838f-f0393d27b738 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:24:26.795316 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application [None req-f5f6f0b4-7f5b-4ea0-838f-f0393d27b738 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 25 05:24:26.795316 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:24:26.795316 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:24:26.795316 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:24:26.795316 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:24:26.795316 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:24:26.795316 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:24:26.795316 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:24:26.795316 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:24:26.795316 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:24:26.795316 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:24:26.795316 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:24:26.795316 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 25 05:24:26.795316 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 25 05:24:26.795316 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 25 05:24:26.795316 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 25 05:24:26.796568 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:24:26.796568 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:24:26.796568 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 25 05:24:26.796568 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 05:24:26.796568 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 25 05:24:26.796568 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 05:24:26.796568 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:24:26.796568 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:24:26.796568 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:24:26.796568 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:24:26.796568 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:24:26.796568 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application with Lock( Jul 25 05:24:26.796568 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:24:26.796568 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:24:26.796568 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:24:26.796568 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:24:26.796568 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:24:26.796568 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:24:26.797829 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:24:26.797829 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:24:26.797829 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 25 05:24:26.797829 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 25 05:24:26.797829 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 25 05:24:26.797829 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 25 05:24:26.797829 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 25 05:24:26.797829 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 25 05:24:26.797829 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 25 05:24:26.797829 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Jul 25 05:24:26.803367 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 332/652] 10.222.0.33 () {56 vars in 1099 bytes} [Mon Jul 25 05:24:26 2022] GET /identity/v3/users/cinder => generated 84 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:24:26.803367 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f6f0b4-7f5b-4ea0-838f-f0393d27b738 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:24:26.852601 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:24:26.885622 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 321/653] 10.222.0.33 () {58 vars in 1282 bytes} [Mon Jul 25 05:24:26 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 422 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 25 05:24:26.887436 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da7e2c1-0eb0-4cf2-ba35-793c7d1e001f None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:24:26.936428 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f6f0b4-7f5b-4ea0-838f-f0393d27b738 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:24:26.966871 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:24:26.981924 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78505e08-018d-4620-81bf-78d10a84a336 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:24:26.982709 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78505e08-018d-4620-81bf-78d10a84a336 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:24:26.983064 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78505e08-018d-4620-81bf-78d10a84a336 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:24:26.983410 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78505e08-018d-4620-81bf-78d10a84a336 None admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:24:26.984253 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78505e08-018d-4620-81bf-78d10a84a336 None admin] RBAC: Adding query filter params (name=service) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:24:26.984653 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78505e08-018d-4620-81bf-78d10a84a336 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:24:26.986654 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78505e08-018d-4620-81bf-78d10a84a336 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:24:26.998908 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 333/654] 10.222.0.33 () {56 vars in 1103 bytes} [Mon Jul 25 05:24:26 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:24:26.999902 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78505e08-018d-4620-81bf-78d10a84a336 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:24:27.019364 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da7e2c1-0eb0-4cf2-ba35-793c7d1e001f None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:24:27.066015 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc47782-ffc7-4304-ba50-c9a7471d3f3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:24:27.067549 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc47782-ffc7-4304-ba50-c9a7471d3f3b None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:24:27.068027 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc47782-ffc7-4304-ba50-c9a7471d3f3b None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:24:27.068825 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc47782-ffc7-4304-ba50-c9a7471d3f3b None admin] PATH_INFO: `/v3/users` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:24:27.070175 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fc47782-ffc7-4304-ba50-c9a7471d3f3b None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:24:27.071128 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fc47782-ffc7-4304-ba50-c9a7471d3f3b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:24:27.073324 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fc47782-ffc7-4304-ba50-c9a7471d3f3b None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:24:27.097911 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 322/655] 10.222.0.33 () {56 vars in 1137 bytes} [Mon Jul 25 05:24:26 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:24:27.099351 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc47782-ffc7-4304-ba50-c9a7471d3f3b None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:24:27.124099 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:24:27.128123 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78505e08-018d-4620-81bf-78d10a84a336 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:24:27.173282 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85935daa-f141-436d-87d9-aeb3ced993f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:24:27.174355 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85935daa-f141-436d-87d9-aeb3ced993f8 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:24:27.174713 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85935daa-f141-436d-87d9-aeb3ced993f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:24:27.175381 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85935daa-f141-436d-87d9-aeb3ced993f8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:24:27.191880 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85935daa-f141-436d-87d9-aeb3ced993f8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:24:27.193657 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85935daa-f141-436d-87d9-aeb3ced993f8 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:24:27.216484 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application [None req-85935daa-f141-436d-87d9-aeb3ced993f8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 05:24:27.216484 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:24:27.216484 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:24:27.216484 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:24:27.216484 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:24:27.216484 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:24:27.216484 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:24:27.216484 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:24:27.216484 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:24:27.216484 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:24:27.216484 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:24:27.216484 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:24:27.216484 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 05:24:27.216484 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 05:24:27.216484 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 05:24:27.217521 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 05:24:27.217521 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:24:27.217521 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:24:27.217521 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:24:27.217521 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:24:27.217521 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:24:27.217521 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:24:27.217521 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:24:27.217521 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application with Lock( Jul 25 05:24:27.217521 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:24:27.217521 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:24:27.217521 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:24:27.217521 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:24:27.217521 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:24:27.217521 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:24:27.217521 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:24:27.217521 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:24:27.218406 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 05:24:27.218406 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 05:24:27.218406 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 05:24:27.218406 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Jul 25 05:24:27.221001 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 334/656] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Jul 25 05:24:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:24:27.221786 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85935daa-f141-436d-87d9-aeb3ced993f8 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:24:27.234379 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc47782-ffc7-4304-ba50-c9a7471d3f3b None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:24:27.257652 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:24:27.272720 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e5ee8d5-76e0-4465-8a76-30375a89788a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:24:27.273541 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5ee8d5-76e0-4465-8a76-30375a89788a None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:24:27.273829 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5ee8d5-76e0-4465-8a76-30375a89788a None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:24:27.274255 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5ee8d5-76e0-4465-8a76-30375a89788a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:24:27.284646 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e5ee8d5-76e0-4465-8a76-30375a89788a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:24:27.286108 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e5ee8d5-76e0-4465-8a76-30375a89788a None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:24:27.305792 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application [None req-6e5ee8d5-76e0-4465-8a76-30375a89788a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 05:24:27.305792 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:24:27.305792 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:24:27.305792 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:24:27.305792 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:24:27.305792 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:24:27.305792 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:24:27.305792 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:24:27.305792 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:24:27.305792 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:24:27.305792 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:24:27.305792 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:24:27.305792 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 05:24:27.305792 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 05:24:27.305792 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 05:24:27.306707 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 05:24:27.306707 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:24:27.306707 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:24:27.306707 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:24:27.306707 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:24:27.306707 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:24:27.306707 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:24:27.306707 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:24:27.306707 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application with Lock( Jul 25 05:24:27.306707 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:24:27.306707 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:24:27.306707 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:24:27.306707 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:24:27.306707 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:24:27.306707 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:24:27.306707 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:24:27.306707 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:24:27.307604 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 05:24:27.307604 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 05:24:27.307604 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 05:24:27.307604 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Jul 25 05:24:27.307604 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 323/657] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Jul 25 05:24:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:24:27.308534 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eecf9913-3f12-4123-99cf-c052dfa6097a None None] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:24:27.308655 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eecf9913-3f12-4123-99cf-c052dfa6097a None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:24:27.308966 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eecf9913-3f12-4123-99cf-c052dfa6097a None None] PATH_INFO: `/` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:24:27.313052 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 324/658] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jul 25 05:24:27 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:24:27.314598 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eecf9913-3f12-4123-99cf-c052dfa6097a None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:24:27.332337 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85935daa-f141-436d-87d9-aeb3ced993f8 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:24:27.371358 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3eee7cb-a2b8-4c27-ab8b-f1a198692e3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:24:27.372468 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3eee7cb-a2b8-4c27-ab8b-f1a198692e3f None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:24:27.373372 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3eee7cb-a2b8-4c27-ab8b-f1a198692e3f None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:24:27.373886 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3eee7cb-a2b8-4c27-ab8b-f1a198692e3f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:24:27.399569 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:24:27.423520 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eecf9913-3f12-4123-99cf-c052dfa6097a None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:24:27.451787 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3eee7cb-a2b8-4c27-ab8b-f1a198692e3f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:24:27.453307 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3eee7cb-a2b8-4c27-ab8b-f1a198692e3f None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:24:27.459551 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4650c45b-a84f-4abc-90f2-bb6cde3ded90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:24:27.460211 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4650c45b-a84f-4abc-90f2-bb6cde3ded90 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:24:27.460495 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4650c45b-a84f-4abc-90f2-bb6cde3ded90 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:24:27.460766 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4650c45b-a84f-4abc-90f2-bb6cde3ded90 None admin] PATH_INFO: `/v3/domains` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:24:27.461925 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4650c45b-a84f-4abc-90f2-bb6cde3ded90 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:24:27.462233 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4650c45b-a84f-4abc-90f2-bb6cde3ded90 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:24:27.463925 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4650c45b-a84f-4abc-90f2-bb6cde3ded90 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:24:27.475939 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 325/659] 10.222.0.33 () {56 vars in 1109 bytes} [Mon Jul 25 05:24:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:24:27.476750 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4650c45b-a84f-4abc-90f2-bb6cde3ded90 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:24:27.509041 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:24:27.569384 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 335/660] 10.222.0.33 () {58 vars in 1282 bytes} [Mon Jul 25 05:24:27 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 344 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 25 05:24:27.569384 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471edc15-9590-4435-87a6-3199f7d01dd4 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:24:27.569384 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471edc15-9590-4435-87a6-3199f7d01dd4 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:24:27.569384 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471edc15-9590-4435-87a6-3199f7d01dd4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:24:27.584158 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-471edc15-9590-4435-87a6-3199f7d01dd4 None None] MFA Rules not processed for user `1eae540242d94c10b0ad33960a855f08`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:24:27.585557 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-471edc15-9590-4435-87a6-3199f7d01dd4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:24:27.589408 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4650c45b-a84f-4abc-90f2-bb6cde3ded90 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:24:27.627247 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8670ecbd-beb4-46d1-9966-50a1856512b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:24:27.627914 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8670ecbd-beb4-46d1-9966-50a1856512b7 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:24:27.628203 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8670ecbd-beb4-46d1-9966-50a1856512b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:24:27.628493 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8670ecbd-beb4-46d1-9966-50a1856512b7 None admin] PATH_INFO: `/v3/domains` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:24:27.629255 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8670ecbd-beb4-46d1-9966-50a1856512b7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:24:27.629568 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8670ecbd-beb4-46d1-9966-50a1856512b7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:24:27.630924 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8670ecbd-beb4-46d1-9966-50a1856512b7 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:24:27.633767 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:24:27.643094 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 326/661] 10.222.0.33 () {56 vars in 1109 bytes} [Mon Jul 25 05:24:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:24:27.643916 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8670ecbd-beb4-46d1-9966-50a1856512b7 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:24:27.668908 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 336/662] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jul 25 05:24:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 05:24:27.668908 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-471edc15-9590-4435-87a6-3199f7d01dd4 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:24:27.745287 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:24:27.755343 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8670ecbd-beb4-46d1-9966-50a1856512b7 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:24:27.783257 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-471edc15-9590-4435-87a6-3199f7d01dd4 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:24:27.791334 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94762362-e9ea-446e-957f-6fbff024e11f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:24:27.792127 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94762362-e9ea-446e-957f-6fbff024e11f None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:24:27.792417 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94762362-e9ea-446e-957f-6fbff024e11f None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:24:27.792787 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94762362-e9ea-446e-957f-6fbff024e11f None admin] PATH_INFO: `/v3/projects/service` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:24:27.805663 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94762362-e9ea-446e-957f-6fbff024e11f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:24:27.807332 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94762362-e9ea-446e-957f-6fbff024e11f None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:24:27.819131 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd506f5-8838-40e5-99f9-f9a6d150ef6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:24:27.819941 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd506f5-8838-40e5-99f9-f9a6d150ef6f None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:24:27.820230 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd506f5-8838-40e5-99f9-f9a6d150ef6f None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:24:27.820513 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd506f5-8838-40e5-99f9-f9a6d150ef6f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:24:27.823496 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application [None req-94762362-e9ea-446e-957f-6fbff024e11f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 05:24:27.823496 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:24:27.823496 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:24:27.823496 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:24:27.823496 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:24:27.823496 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:24:27.823496 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:24:27.823496 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:24:27.823496 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:24:27.823496 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:24:27.823496 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:24:27.823496 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:24:27.823496 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 25 05:24:27.823496 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 25 05:24:27.823496 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 25 05:24:27.825221 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 25 05:24:27.825221 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:24:27.825221 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:24:27.825221 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:24:27.825221 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:24:27.825221 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:24:27.825221 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:24:27.825221 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:24:27.825221 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application with Lock( Jul 25 05:24:27.825221 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:24:27.825221 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:24:27.825221 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:24:27.825221 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:24:27.825221 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:24:27.825221 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:24:27.825221 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:24:27.825221 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:24:27.826307 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 25 05:24:27.826307 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 25 05:24:27.826307 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 25 05:24:27.826307 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 25 05:24:27.826307 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 25 05:24:27.826307 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 25 05:24:27.826307 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 05:24:27.826307 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Jul 25 05:24:27.828566 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 327/663] 10.222.0.33 () {56 vars in 1111 bytes} [Mon Jul 25 05:24:27 2022] GET /identity/v3/projects/service => generated 88 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:24:27.829636 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94762362-e9ea-446e-957f-6fbff024e11f None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:24:27.838710 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bd506f5-8838-40e5-99f9-f9a6d150ef6f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:24:27.840527 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bd506f5-8838-40e5-99f9-f9a6d150ef6f None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:24:27.860411 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application [None req-5bd506f5-8838-40e5-99f9-f9a6d150ef6f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 05:24:27.860411 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:24:27.860411 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:24:27.860411 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:24:27.860411 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:24:27.860411 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:24:27.860411 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:24:27.860411 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:24:27.860411 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:24:27.860411 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:24:27.860411 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:24:27.860411 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:24:27.860411 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 05:24:27.860411 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 05:24:27.860411 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 05:24:27.861667 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 05:24:27.861667 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:24:27.861667 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:24:27.861667 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:24:27.861667 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:24:27.861667 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:24:27.861667 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:24:27.861667 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:24:27.861667 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application with Lock( Jul 25 05:24:27.861667 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:24:27.861667 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:24:27.861667 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:24:27.861667 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:24:27.861667 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:24:27.861667 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:24:27.861667 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:24:27.861667 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:24:27.862765 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 05:24:27.862765 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 05:24:27.862765 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 05:24:27.862765 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Jul 25 05:24:27.862765 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 337/664] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Jul 25 05:24:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:24:27.862765 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd506f5-8838-40e5-99f9-f9a6d150ef6f None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:24:27.898616 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:24:27.947547 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94762362-e9ea-446e-957f-6fbff024e11f None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:24:27.974216 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd506f5-8838-40e5-99f9-f9a6d150ef6f None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:24:27.988987 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48904b6b-42d2-4c61-ae25-77d074f25362 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:24:27.989781 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48904b6b-42d2-4c61-ae25-77d074f25362 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:24:27.990067 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48904b6b-42d2-4c61-ae25-77d074f25362 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:24:27.990348 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48904b6b-42d2-4c61-ae25-77d074f25362 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:24:27.999508 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48904b6b-42d2-4c61-ae25-77d074f25362 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:24:28.000992 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48904b6b-42d2-4c61-ae25-77d074f25362 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:24:28.010673 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e41fab6-699e-4c2e-84b6-e65b35abad9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:24:28.011747 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e41fab6-699e-4c2e-84b6-e65b35abad9d None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:24:28.012391 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e41fab6-699e-4c2e-84b6-e65b35abad9d None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:24:28.012918 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e41fab6-699e-4c2e-84b6-e65b35abad9d None admin] PATH_INFO: `/v3/roles/service` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:24:28.015342 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application [None req-48904b6b-42d2-4c61-ae25-77d074f25362 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jul 25 05:24:28.015342 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:24:28.015342 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:24:28.015342 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:24:28.015342 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:24:28.015342 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:24:28.015342 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:24:28.015342 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:24:28.015342 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:24:28.015342 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:24:28.015342 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:24:28.015342 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:24:28.015342 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 25 05:24:28.015342 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 25 05:24:28.015342 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 25 05:24:28.015342 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 25 05:24:28.016275 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:24:28.016275 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:24:28.016275 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 25 05:24:28.016275 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 05:24:28.016275 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 25 05:24:28.016275 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 05:24:28.016275 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:24:28.016275 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:24:28.016275 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:24:28.016275 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:24:28.016275 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:24:28.016275 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application with Lock( Jul 25 05:24:28.016275 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:24:28.016275 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:24:28.016275 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:24:28.016275 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:24:28.016275 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:24:28.016275 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:24:28.017290 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:24:28.017290 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:24:28.017290 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 25 05:24:28.017290 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 25 05:24:28.017290 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 25 05:24:28.017290 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 25 05:24:28.017290 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 25 05:24:28.017290 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 25 05:24:28.017290 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jul 25 05:24:28.017290 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Jul 25 05:24:28.017290 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 328/665] 10.222.0.33 () {56 vars in 1102 bytes} [Mon Jul 25 05:24:27 2022] GET /identity/v3/users/neutron => generated 85 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:24:28.017290 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48904b6b-42d2-4c61-ae25-77d074f25362 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:24:28.031035 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e41fab6-699e-4c2e-84b6-e65b35abad9d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 05:24:28.033675 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:24:28.040663 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application [None req-0e41fab6-699e-4c2e-84b6-e65b35abad9d None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 25 05:24:28.040663 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:24:28.040663 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:24:28.040663 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:24:28.040663 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:24:28.040663 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:24:28.040663 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:24:28.040663 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:24:28.040663 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:24:28.040663 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:24:28.040663 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:24:28.040663 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:24:28.040663 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 05:24:28.040663 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 05:24:28.040663 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 05:24:28.040663 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 05:24:28.041788 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 05:24:28.041788 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 05:24:28.041788 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 05:24:28.041788 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 05:24:28.041788 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:24:28.041788 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:24:28.041788 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:24:28.041788 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:24:28.041788 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:24:28.041788 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:24:28.041788 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:24:28.041788 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application with Lock( Jul 25 05:24:28.041788 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:24:28.041788 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:24:28.041788 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:24:28.041788 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:24:28.043228 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:24:28.043228 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:24:28.043228 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:24:28.043228 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:24:28.043228 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 05:24:28.043228 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 05:24:28.043228 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 05:24:28.043228 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 05:24:28.043228 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 05:24:28.043228 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 05:24:28.043228 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 25 05:24:28.043228 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Jul 25 05:24:28.043228 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 338/666] 10.222.0.33 () {56 vars in 1102 bytes} [Mon Jul 25 05:24:27 2022] GET /identity/v3/roles/service => generated 85 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:24:28.044536 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e41fab6-699e-4c2e-84b6-e65b35abad9d None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:24:28.124796 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48904b6b-42d2-4c61-ae25-77d074f25362 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:24:28.143305 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:24:28.151603 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e41fab6-699e-4c2e-84b6-e65b35abad9d None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:24:28.157735 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e6c625a-bc62-47ce-a8d2-487eb89aeb1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:24:28.158437 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6c625a-bc62-47ce-a8d2-487eb89aeb1a None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:24:28.158697 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6c625a-bc62-47ce-a8d2-487eb89aeb1a None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:24:28.158946 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6c625a-bc62-47ce-a8d2-487eb89aeb1a None admin] PATH_INFO: `/v3/projects` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:24:28.159741 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e6c625a-bc62-47ce-a8d2-487eb89aeb1a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:24:28.160053 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e6c625a-bc62-47ce-a8d2-487eb89aeb1a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:24:28.161516 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e6c625a-bc62-47ce-a8d2-487eb89aeb1a None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:24:28.174093 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 329/667] 10.222.0.33 () {56 vars in 1148 bytes} [Mon Jul 25 05:24:28 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:24:28.174908 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e6c625a-bc62-47ce-a8d2-487eb89aeb1a None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:24:28.193173 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d72429-165d-4c33-aa1b-9ab90144399b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:24:28.194169 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d72429-165d-4c33-aa1b-9ab90144399b None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:24:28.194608 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d72429-165d-4c33-aa1b-9ab90144399b None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:24:28.195103 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d72429-165d-4c33-aa1b-9ab90144399b None admin] PATH_INFO: `/v3/domains` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:24:28.195984 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04d72429-165d-4c33-aa1b-9ab90144399b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:24:28.196464 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04d72429-165d-4c33-aa1b-9ab90144399b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:24:28.198213 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04d72429-165d-4c33-aa1b-9ab90144399b None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:24:28.212308 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 339/668] 10.222.0.33 () {56 vars in 1109 bytes} [Mon Jul 25 05:24:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:24:28.213379 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d72429-165d-4c33-aa1b-9ab90144399b None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:24:28.247925 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:24:28.279057 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e6c625a-bc62-47ce-a8d2-487eb89aeb1a None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:24:28.314423 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec347ea-a633-4b9b-9020-3c2b8689658e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:24:28.315097 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec347ea-a633-4b9b-9020-3c2b8689658e None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:24:28.315378 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec347ea-a633-4b9b-9020-3c2b8689658e None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:24:28.315651 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec347ea-a633-4b9b-9020-3c2b8689658e None admin] PATH_INFO: `/v3/users` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:24:28.316508 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ec347ea-a633-4b9b-9020-3c2b8689658e None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:24:28.316820 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ec347ea-a633-4b9b-9020-3c2b8689658e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:24:28.318248 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ec347ea-a633-4b9b-9020-3c2b8689658e None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:24:28.321778 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d72429-165d-4c33-aa1b-9ab90144399b None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:24:28.343097 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 330/669] 10.222.0.33 () {56 vars in 1139 bytes} [Mon Jul 25 05:24:28 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:24:28.344079 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec347ea-a633-4b9b-9020-3c2b8689658e None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:24:28.363862 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c90c202-4747-4f21-b03d-5f158edc7111 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:24:28.364538 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c90c202-4747-4f21-b03d-5f158edc7111 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:24:28.364836 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c90c202-4747-4f21-b03d-5f158edc7111 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:24:28.365118 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c90c202-4747-4f21-b03d-5f158edc7111 None admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:24:28.365875 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c90c202-4747-4f21-b03d-5f158edc7111 None admin] RBAC: Adding query filter params (name=service) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:24:28.366178 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c90c202-4747-4f21-b03d-5f158edc7111 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:24:28.367552 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c90c202-4747-4f21-b03d-5f158edc7111 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:24:28.377598 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 340/670] 10.222.0.33 () {56 vars in 1103 bytes} [Mon Jul 25 05:24:28 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:24:28.378402 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c90c202-4747-4f21-b03d-5f158edc7111 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:24:28.399760 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:24:28.454295 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec347ea-a633-4b9b-9020-3c2b8689658e None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:24:28.483399 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:24:28.489712 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b925f0d9-e10d-47a5-b77d-53525ffd0db4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:24:28.490341 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c90c202-4747-4f21-b03d-5f158edc7111 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:24:28.490447 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b925f0d9-e10d-47a5-b77d-53525ffd0db4 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:24:28.490639 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b925f0d9-e10d-47a5-b77d-53525ffd0db4 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:24:28.490906 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b925f0d9-e10d-47a5-b77d-53525ffd0db4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:24:28.491738 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b925f0d9-e10d-47a5-b77d-53525ffd0db4 None admin] RBAC: Adding query filter params (role.id=200e95cafbc54269a51756ddd406082f, scope.project.id=92afba266cce4d1bad4134c351e74be3, user.id=1cd482a61e8a420db45bb532e5dae6e2) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:24:28.492048 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b925f0d9-e10d-47a5-b77d-53525ffd0db4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:24:28.493495 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b925f0d9-e10d-47a5-b77d-53525ffd0db4 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:24:28.509230 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 331/671] 10.222.0.33 () {56 vars in 1374 bytes} [Mon Jul 25 05:24:28 2022] GET /identity/v3/role_assignments?user.id=1cd482a61e8a420db45bb532e5dae6e2&scope.project.id=92afba266cce4d1bad4134c351e74be3&role.id=200e95cafbc54269a51756ddd406082f => generated 608 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:24:28.510571 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b925f0d9-e10d-47a5-b77d-53525ffd0db4 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:24:28.537852 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa03d3b4-44cf-4986-bcea-df4d427e2990 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:24:28.539156 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa03d3b4-44cf-4986-bcea-df4d427e2990 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:24:28.539714 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa03d3b4-44cf-4986-bcea-df4d427e2990 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:24:28.540232 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa03d3b4-44cf-4986-bcea-df4d427e2990 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:24:28.550490 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa03d3b4-44cf-4986-bcea-df4d427e2990 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:24:28.552409 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa03d3b4-44cf-4986-bcea-df4d427e2990 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:24:28.570497 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application [None req-fa03d3b4-44cf-4986-bcea-df4d427e2990 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 05:24:28.570497 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:24:28.570497 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:24:28.570497 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:24:28.570497 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:24:28.570497 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:24:28.570497 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:24:28.570497 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:24:28.570497 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:24:28.570497 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:24:28.570497 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:24:28.570497 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:24:28.570497 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 25 05:24:28.570497 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 25 05:24:28.570497 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 25 05:24:28.571470 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 25 05:24:28.571470 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:24:28.571470 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:24:28.571470 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:24:28.571470 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:24:28.571470 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:24:28.571470 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:24:28.571470 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:24:28.571470 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application with Lock( Jul 25 05:24:28.571470 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:24:28.571470 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:24:28.571470 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:24:28.571470 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:24:28.571470 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:24:28.571470 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:24:28.571470 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:24:28.571470 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:24:28.572361 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 25 05:24:28.572361 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 25 05:24:28.572361 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 25 05:24:28.572361 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 25 05:24:28.572361 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 25 05:24:28.572361 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 25 05:24:28.572361 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 05:24:28.572361 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Jul 25 05:24:28.575899 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 341/672] 10.222.0.33 () {56 vars in 1111 bytes} [Mon Jul 25 05:24:28 2022] GET /identity/v3/projects/service => generated 88 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:24:28.577156 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa03d3b4-44cf-4986-bcea-df4d427e2990 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:24:28.628294 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b925f0d9-e10d-47a5-b77d-53525ffd0db4 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:24:28.670121 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35fd5657-f2f0-4cd2-b9f5-22b1ef550689 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:24:28.670937 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35fd5657-f2f0-4cd2-b9f5-22b1ef550689 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:24:28.671339 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35fd5657-f2f0-4cd2-b9f5-22b1ef550689 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:24:28.671714 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35fd5657-f2f0-4cd2-b9f5-22b1ef550689 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:24:28.672552 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:24:28.707055 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa03d3b4-44cf-4986-bcea-df4d427e2990 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:24:28.749860 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1426bba-a491-490d-a27f-242514e6c5d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:24:28.750817 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1426bba-a491-490d-a27f-242514e6c5d3 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:24:28.751201 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1426bba-a491-490d-a27f-242514e6c5d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:24:28.751582 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1426bba-a491-490d-a27f-242514e6c5d3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:24:28.753539 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:24:28.763192 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35fd5657-f2f0-4cd2-b9f5-22b1ef550689 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:24:28.764805 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35fd5657-f2f0-4cd2-b9f5-22b1ef550689 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:24:28.768497 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1426bba-a491-490d-a27f-242514e6c5d3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:24:28.769435 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1426bba-a491-490d-a27f-242514e6c5d3 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:24:28.791239 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application [None req-f1426bba-a491-490d-a27f-242514e6c5d3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 05:24:28.791239 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:24:28.791239 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:24:28.791239 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:24:28.791239 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:24:28.791239 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:24:28.791239 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:24:28.791239 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:24:28.791239 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:24:28.791239 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:24:28.791239 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:24:28.791239 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:24:28.791239 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 05:24:28.791239 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 05:24:28.791239 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 05:24:28.792364 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 05:24:28.792364 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:24:28.792364 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:24:28.792364 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:24:28.792364 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:24:28.792364 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:24:28.792364 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:24:28.792364 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:24:28.792364 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application with Lock( Jul 25 05:24:28.792364 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:24:28.792364 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:24:28.792364 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:24:28.792364 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:24:28.792364 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:24:28.792364 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:24:28.792364 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:24:28.792364 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:24:28.793901 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 05:24:28.793901 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 05:24:28.793901 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 05:24:28.793901 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Jul 25 05:24:28.793901 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 342/673] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Jul 25 05:24:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:24:28.794409 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1426bba-a491-490d-a27f-242514e6c5d3 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:24:28.882832 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 332/674] 10.222.0.33 () {58 vars in 1282 bytes} [Mon Jul 25 05:24:28 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 25 05:24:28.889670 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7eb7462-a867-47dd-94cf-e1f74192a086 None None] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:24:28.889962 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7eb7462-a867-47dd-94cf-e1f74192a086 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:24:28.890245 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7eb7462-a867-47dd-94cf-e1f74192a086 None None] PATH_INFO: `/` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:24:28.891557 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 333/675] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jul 25 05:24:28 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:24:28.892247 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7eb7462-a867-47dd-94cf-e1f74192a086 None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:24:28.916078 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1426bba-a491-490d-a27f-242514e6c5d3 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:24:28.945259 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:24:28.955318 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8be2d4d-96f0-4676-9eaf-7811c5819cda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:24:28.956085 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8be2d4d-96f0-4676-9eaf-7811c5819cda None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:24:28.956613 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8be2d4d-96f0-4676-9eaf-7811c5819cda None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:24:28.956830 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8be2d4d-96f0-4676-9eaf-7811c5819cda None admin] PATH_INFO: `/v3/projects` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:24:28.957681 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8be2d4d-96f0-4676-9eaf-7811c5819cda None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:24:28.957940 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8be2d4d-96f0-4676-9eaf-7811c5819cda None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:24:28.959602 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8be2d4d-96f0-4676-9eaf-7811c5819cda None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:24:28.978276 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 343/676] 10.222.0.33 () {56 vars in 1148 bytes} [Mon Jul 25 05:24:28 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:24:28.978276 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8be2d4d-96f0-4676-9eaf-7811c5819cda None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:24:28.996392 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:24:29.007283 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7eb7462-a867-47dd-94cf-e1f74192a086 None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:24:29.044646 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d44a0488-32ae-450d-abd0-d1fc16da5878 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:24:29.045345 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44a0488-32ae-450d-abd0-d1fc16da5878 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:24:29.045637 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44a0488-32ae-450d-abd0-d1fc16da5878 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:24:29.045911 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44a0488-32ae-450d-abd0-d1fc16da5878 None admin] PATH_INFO: `/v3/domains` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:24:29.046642 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d44a0488-32ae-450d-abd0-d1fc16da5878 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:24:29.046957 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d44a0488-32ae-450d-abd0-d1fc16da5878 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:24:29.048315 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d44a0488-32ae-450d-abd0-d1fc16da5878 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:24:29.060135 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 334/677] 10.222.0.33 () {56 vars in 1109 bytes} [Mon Jul 25 05:24:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:24:29.061403 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b921e01-92ca-48ec-a53a-55a5e919fda7 None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:24:29.061752 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b921e01-92ca-48ec-a53a-55a5e919fda7 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:24:29.062080 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b921e01-92ca-48ec-a53a-55a5e919fda7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:24:29.078672 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-6b921e01-92ca-48ec-a53a-55a5e919fda7 None None] MFA Rules not processed for user `1eae540242d94c10b0ad33960a855f08`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:24:29.079865 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-6b921e01-92ca-48ec-a53a-55a5e919fda7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:24:29.093374 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8be2d4d-96f0-4676-9eaf-7811c5819cda None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:24:29.131446 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa512b9-bad0-4a80-a159-971b4d77b980 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:24:29.132533 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa512b9-bad0-4a80-a159-971b4d77b980 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:24:29.133058 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa512b9-bad0-4a80-a159-971b4d77b980 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:24:29.133482 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa512b9-bad0-4a80-a159-971b4d77b980 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:24:29.144848 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caa512b9-bad0-4a80-a159-971b4d77b980 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:24:29.146490 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caa512b9-bad0-4a80-a159-971b4d77b980 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:24:29.151737 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 335/678] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jul 25 05:24:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 05:24:29.153140 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b921e01-92ca-48ec-a53a-55a5e919fda7 None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:24:29.176136 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application [None req-caa512b9-bad0-4a80-a159-971b4d77b980 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 05:24:29.176136 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:24:29.176136 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:24:29.176136 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:24:29.176136 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:24:29.176136 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:24:29.176136 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:24:29.176136 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:24:29.176136 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:24:29.176136 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:24:29.176136 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:24:29.176136 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:24:29.176136 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 05:24:29.176136 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 05:24:29.176136 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 05:24:29.177140 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 05:24:29.177140 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:24:29.177140 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:24:29.177140 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:24:29.177140 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:24:29.177140 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:24:29.177140 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:24:29.177140 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:24:29.177140 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application with Lock( Jul 25 05:24:29.177140 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:24:29.177140 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:24:29.177140 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:24:29.177140 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:24:29.177140 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:24:29.177140 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:24:29.177140 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:24:29.177140 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:24:29.178100 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 05:24:29.178100 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 05:24:29.178100 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 05:24:29.178100 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Jul 25 05:24:29.185163 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 344/679] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Jul 25 05:24:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:24:29.186163 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa512b9-bad0-4a80-a159-971b4d77b980 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:24:29.229862 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:24:29.246478 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:24:29.265662 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b921e01-92ca-48ec-a53a-55a5e919fda7 None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:24:29.296171 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa512b9-bad0-4a80-a159-971b4d77b980 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:24:29.301561 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a143f06-5448-40b1-93aa-f3ec067d69bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:24:29.302317 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a143f06-5448-40b1-93aa-f3ec067d69bd None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:24:29.302674 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a143f06-5448-40b1-93aa-f3ec067d69bd None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:24:29.303022 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a143f06-5448-40b1-93aa-f3ec067d69bd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:24:29.303937 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a143f06-5448-40b1-93aa-f3ec067d69bd None admin] RBAC: Adding query filter params (role.id=200e95cafbc54269a51756ddd406082f, scope.project.id=92afba266cce4d1bad4134c351e74be3, user.id=db57b478c25643dbaf936f90f6e17d0a) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:24:29.304322 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a143f06-5448-40b1-93aa-f3ec067d69bd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:24:29.305878 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a143f06-5448-40b1-93aa-f3ec067d69bd None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:24:29.320813 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 336/680] 10.222.0.33 () {56 vars in 1374 bytes} [Mon Jul 25 05:24:29 2022] GET /identity/v3/role_assignments?user.id=db57b478c25643dbaf936f90f6e17d0a&scope.project.id=92afba266cce4d1bad4134c351e74be3&role.id=200e95cafbc54269a51756ddd406082f => generated 608 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:24:29.320813 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a143f06-5448-40b1-93aa-f3ec067d69bd None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:24:29.342787 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-188da431-98c5-415c-932f-7ab39a0c950b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:24:29.343722 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188da431-98c5-415c-932f-7ab39a0c950b None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:24:29.344077 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188da431-98c5-415c-932f-7ab39a0c950b None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:24:29.344424 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188da431-98c5-415c-932f-7ab39a0c950b None admin] PATH_INFO: `/v3/users/swift` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:24:29.353735 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-188da431-98c5-415c-932f-7ab39a0c950b None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:24:29.356007 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-188da431-98c5-415c-932f-7ab39a0c950b None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:24:29.373178 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application [None req-188da431-98c5-415c-932f-7ab39a0c950b None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 25 05:24:29.373178 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:24:29.373178 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:24:29.373178 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:24:29.373178 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:24:29.373178 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:24:29.373178 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:24:29.373178 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:24:29.373178 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:24:29.373178 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:24:29.373178 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:24:29.373178 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:24:29.373178 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 25 05:24:29.373178 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 25 05:24:29.373178 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 25 05:24:29.373178 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 25 05:24:29.374826 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:24:29.374826 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:24:29.374826 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 25 05:24:29.374826 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 05:24:29.374826 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 25 05:24:29.374826 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 05:24:29.374826 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:24:29.374826 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:24:29.374826 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:24:29.374826 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:24:29.374826 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:24:29.374826 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application with Lock( Jul 25 05:24:29.374826 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:24:29.374826 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:24:29.374826 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:24:29.374826 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:24:29.374826 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:24:29.374826 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:24:29.375978 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:24:29.375978 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:24:29.375978 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 25 05:24:29.375978 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 25 05:24:29.375978 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 25 05:24:29.375978 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 25 05:24:29.375978 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 25 05:24:29.375978 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 25 05:24:29.375978 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 25 05:24:29.375978 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Jul 25 05:24:29.380922 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 345/681] 10.222.0.33 () {56 vars in 1096 bytes} [Mon Jul 25 05:24:29 2022] GET /identity/v3/users/swift => generated 83 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:24:29.381889 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-188da431-98c5-415c-932f-7ab39a0c950b None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:24:29.445199 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a143f06-5448-40b1-93aa-f3ec067d69bd None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:24:29.494955 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d133f83f-c30b-4873-b1a2-dc4d38f17444 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:24:29.494955 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d133f83f-c30b-4873-b1a2-dc4d38f17444 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:24:29.494955 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d133f83f-c30b-4873-b1a2-dc4d38f17444 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:24:29.495374 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d133f83f-c30b-4873-b1a2-dc4d38f17444 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:24:29.510379 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d133f83f-c30b-4873-b1a2-dc4d38f17444 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:24:29.512736 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d133f83f-c30b-4873-b1a2-dc4d38f17444 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:24:29.515905 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:24:29.519496 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-188da431-98c5-415c-932f-7ab39a0c950b None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:24:29.522528 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:24:29.534395 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application [None req-d133f83f-c30b-4873-b1a2-dc4d38f17444 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 05:24:29.534395 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:24:29.534395 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:24:29.534395 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:24:29.534395 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:24:29.534395 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:24:29.534395 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:24:29.534395 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:24:29.534395 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:24:29.534395 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:24:29.534395 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:24:29.534395 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:24:29.534395 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 05:24:29.534395 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 05:24:29.534395 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 05:24:29.535791 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 05:24:29.535791 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:24:29.535791 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:24:29.535791 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:24:29.535791 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:24:29.535791 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:24:29.535791 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:24:29.535791 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:24:29.535791 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application with Lock( Jul 25 05:24:29.535791 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:24:29.535791 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:24:29.535791 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:24:29.535791 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:24:29.535791 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:24:29.535791 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:24:29.535791 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:24:29.535791 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:24:29.537000 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 05:24:29.537000 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 05:24:29.537000 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 05:24:29.537000 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Jul 25 05:24:29.541409 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 337/682] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Jul 25 05:24:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:24:29.541409 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d133f83f-c30b-4873-b1a2-dc4d38f17444 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:24:29.558492 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-580db689-b8fd-4208-b531-467719da2c47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:24:29.560279 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580db689-b8fd-4208-b531-467719da2c47 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:24:29.560760 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580db689-b8fd-4208-b531-467719da2c47 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:24:29.561211 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580db689-b8fd-4208-b531-467719da2c47 None admin] PATH_INFO: `/v3/domains` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:24:29.562194 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-580db689-b8fd-4208-b531-467719da2c47 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:24:29.563406 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-580db689-b8fd-4208-b531-467719da2c47 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:24:29.565391 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-580db689-b8fd-4208-b531-467719da2c47 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:24:29.578142 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 346/683] 10.222.0.33 () {56 vars in 1109 bytes} [Mon Jul 25 05:24:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:24:29.578810 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-580db689-b8fd-4208-b531-467719da2c47 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:24:29.681563 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d133f83f-c30b-4873-b1a2-dc4d38f17444 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:24:29.694183 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-580db689-b8fd-4208-b531-467719da2c47 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:24:29.730448 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ede87e-130b-489d-a8fa-9db6409371aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:24:29.731149 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ede87e-130b-489d-a8fa-9db6409371aa None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:24:29.731438 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ede87e-130b-489d-a8fa-9db6409371aa None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:24:29.731879 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ede87e-130b-489d-a8fa-9db6409371aa None admin] PATH_INFO: `/v3/users` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:24:29.733075 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8ede87e-130b-489d-a8fa-9db6409371aa None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:24:29.733876 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8ede87e-130b-489d-a8fa-9db6409371aa None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:24:29.735664 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8ede87e-130b-489d-a8fa-9db6409371aa None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:24:29.742681 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2455c88-0950-4bf4-b266-13c6af2f7b19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:24:29.744366 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2455c88-0950-4bf4-b266-13c6af2f7b19 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:24:29.744890 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2455c88-0950-4bf4-b266-13c6af2f7b19 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:24:29.745261 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2455c88-0950-4bf4-b266-13c6af2f7b19 None admin] PATH_INFO: `/v3/domains` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:24:29.747507 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2455c88-0950-4bf4-b266-13c6af2f7b19 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:24:29.748407 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2455c88-0950-4bf4-b266-13c6af2f7b19 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:24:29.750070 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2455c88-0950-4bf4-b266-13c6af2f7b19 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:24:29.760885 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 338/684] 10.222.0.33 () {56 vars in 1135 bytes} [Mon Jul 25 05:24:29 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:24:29.762760 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ede87e-130b-489d-a8fa-9db6409371aa None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:24:29.777221 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 347/685] 10.222.0.33 () {56 vars in 1109 bytes} [Mon Jul 25 05:24:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:24:29.778299 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2455c88-0950-4bf4-b266-13c6af2f7b19 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:24:29.807730 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:24:29.845389 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:24:29.888379 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ede87e-130b-489d-a8fa-9db6409371aa None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:24:29.905080 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2455c88-0950-4bf4-b266-13c6af2f7b19 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:24:29.924799 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c14ed65-232f-47d2-88cc-896d8c6b3fa2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:24:29.925605 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c14ed65-232f-47d2-88cc-896d8c6b3fa2 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:24:29.925889 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c14ed65-232f-47d2-88cc-896d8c6b3fa2 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:24:29.926166 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c14ed65-232f-47d2-88cc-896d8c6b3fa2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:24:29.933762 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c14ed65-232f-47d2-88cc-896d8c6b3fa2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:24:29.935686 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c14ed65-232f-47d2-88cc-896d8c6b3fa2 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:24:29.946765 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be07a7c-b2bf-4b9d-acca-d36e70fd9599 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:24:29.947673 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be07a7c-b2bf-4b9d-acca-d36e70fd9599 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:24:29.948166 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be07a7c-b2bf-4b9d-acca-d36e70fd9599 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:24:29.948668 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be07a7c-b2bf-4b9d-acca-d36e70fd9599 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:24:29.949897 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application [None req-0c14ed65-232f-47d2-88cc-896d8c6b3fa2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 05:24:29.949897 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:24:29.949897 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:24:29.949897 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:24:29.949897 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:24:29.949897 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:24:29.949897 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:24:29.949897 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:24:29.949897 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:24:29.949897 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:24:29.949897 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:24:29.949897 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:24:29.949897 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 25 05:24:29.949897 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 25 05:24:29.949897 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 25 05:24:29.950766 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 25 05:24:29.950766 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:24:29.950766 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:24:29.950766 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:24:29.950766 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:24:29.950766 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:24:29.950766 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:24:29.950766 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:24:29.950766 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application with Lock( Jul 25 05:24:29.950766 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:24:29.950766 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:24:29.950766 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:24:29.950766 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:24:29.950766 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:24:29.950766 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:24:29.950766 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:24:29.950766 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:24:29.951681 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 25 05:24:29.951681 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 25 05:24:29.951681 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 25 05:24:29.951681 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 25 05:24:29.951681 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 25 05:24:29.951681 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 25 05:24:29.951681 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 05:24:29.951681 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Jul 25 05:24:29.955890 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 339/686] 10.222.0.33 () {56 vars in 1111 bytes} [Mon Jul 25 05:24:29 2022] GET /identity/v3/projects/service => generated 88 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:24:29.956719 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c14ed65-232f-47d2-88cc-896d8c6b3fa2 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:24:30.037122 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7be07a7c-b2bf-4b9d-acca-d36e70fd9599 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:24:30.039563 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7be07a7c-b2bf-4b9d-acca-d36e70fd9599 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:24:30.060492 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c14ed65-232f-47d2-88cc-896d8c6b3fa2 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:24:30.068533 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:24:30.079204 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:24:30.098352 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-065d6da5-e372-4434-9e81-146da6896032 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:24:30.099095 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065d6da5-e372-4434-9e81-146da6896032 None admin] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:24:30.099415 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065d6da5-e372-4434-9e81-146da6896032 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:24:30.099663 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065d6da5-e372-4434-9e81-146da6896032 None admin] PATH_INFO: `/v3/users` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:24:30.100378 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-065d6da5-e372-4434-9e81-146da6896032 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:24:30.102014 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-065d6da5-e372-4434-9e81-146da6896032 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:24:30.128939 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 340/687] 10.222.0.33 () {60 vars in 1132 bytes} [Mon Jul 25 05:24:29 2022] POST /identity/v3/users => generated 254 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 05:24:30.130273 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-065d6da5-e372-4434-9e81-146da6896032 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:24:30.211453 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 348/688] 10.222.0.33 () {58 vars in 1282 bytes} [Mon Jul 25 05:24:29 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 433 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 25 05:24:30.221891 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be07a7c-b2bf-4b9d-acca-d36e70fd9599 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:24:30.332367 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-065d6da5-e372-4434-9e81-146da6896032 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:24:30.372818 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d2b31f8-1ea1-4033-90b0-f0b462640d35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:24:30.374039 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2b31f8-1ea1-4033-90b0-f0b462640d35 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:24:30.374562 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2b31f8-1ea1-4033-90b0-f0b462640d35 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:24:30.375062 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2b31f8-1ea1-4033-90b0-f0b462640d35 None admin] PATH_INFO: `/v3/projects` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:24:30.376179 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d2b31f8-1ea1-4033-90b0-f0b462640d35 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:24:30.376813 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d2b31f8-1ea1-4033-90b0-f0b462640d35 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:24:30.378465 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d2b31f8-1ea1-4033-90b0-f0b462640d35 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:24:30.382679 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be07a7c-b2bf-4b9d-acca-d36e70fd9599 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:24:30.393063 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 341/689] 10.222.0.33 () {56 vars in 1148 bytes} [Mon Jul 25 05:24:30 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:24:30.400474 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d2b31f8-1ea1-4033-90b0-f0b462640d35 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:24:30.412325 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:24:30.433163 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8576e52-bbbb-4d69-8425-b05d9ad692a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:24:30.434996 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8576e52-bbbb-4d69-8425-b05d9ad692a6 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:24:30.435504 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8576e52-bbbb-4d69-8425-b05d9ad692a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:24:30.436602 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8576e52-bbbb-4d69-8425-b05d9ad692a6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:24:30.455073 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8576e52-bbbb-4d69-8425-b05d9ad692a6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:24:30.456525 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8576e52-bbbb-4d69-8425-b05d9ad692a6 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:24:30.466342 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:24:30.477869 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application [None req-d8576e52-bbbb-4d69-8425-b05d9ad692a6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 05:24:30.477869 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:24:30.477869 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:24:30.477869 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:24:30.477869 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:24:30.477869 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:24:30.477869 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:24:30.477869 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:24:30.477869 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:24:30.477869 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:24:30.477869 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:24:30.477869 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:24:30.477869 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 05:24:30.477869 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 05:24:30.477869 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 05:24:30.478829 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 05:24:30.478829 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:24:30.478829 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:24:30.478829 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:24:30.478829 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:24:30.478829 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:24:30.478829 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:24:30.478829 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:24:30.478829 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application with Lock( Jul 25 05:24:30.478829 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:24:30.478829 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:24:30.478829 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:24:30.478829 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:24:30.478829 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:24:30.478829 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:24:30.478829 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:24:30.478829 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:24:30.479741 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 05:24:30.479741 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 05:24:30.479741 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 05:24:30.479741 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Jul 25 05:24:30.479741 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 349/690] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Jul 25 05:24:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 258 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:24:30.487958 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8576e52-bbbb-4d69-8425-b05d9ad692a6 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:24:30.551548 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d2b31f8-1ea1-4033-90b0-f0b462640d35 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:24:30.591419 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54313ffd-74ea-4998-ad8a-03aa7d3eb544 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:24:30.592177 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54313ffd-74ea-4998-ad8a-03aa7d3eb544 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:24:30.592509 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54313ffd-74ea-4998-ad8a-03aa7d3eb544 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:24:30.592804 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54313ffd-74ea-4998-ad8a-03aa7d3eb544 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:24:30.593776 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54313ffd-74ea-4998-ad8a-03aa7d3eb544 None admin] RBAC: Adding query filter params (role.id=200e95cafbc54269a51756ddd406082f, scope.project.id=92afba266cce4d1bad4134c351e74be3, user.id=b2931200f99e43d79a50a9f6e02738e6) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:24:30.594093 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54313ffd-74ea-4998-ad8a-03aa7d3eb544 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:24:30.595649 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54313ffd-74ea-4998-ad8a-03aa7d3eb544 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:24:30.600345 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8576e52-bbbb-4d69-8425-b05d9ad692a6 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:24:30.602727 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 342/691] 10.222.0.33 () {56 vars in 1374 bytes} [Mon Jul 25 05:24:30 2022] GET /identity/v3/role_assignments?user.id=b2931200f99e43d79a50a9f6e02738e6&scope.project.id=92afba266cce4d1bad4134c351e74be3&role.id=200e95cafbc54269a51756ddd406082f => generated 608 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:24:30.640605 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50958df3-3952-4902-b47a-c58e94d20f2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:24:30.641316 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50958df3-3952-4902-b47a-c58e94d20f2a None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:24:30.641602 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50958df3-3952-4902-b47a-c58e94d20f2a None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:24:30.641880 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50958df3-3952-4902-b47a-c58e94d20f2a None admin] PATH_INFO: `/v3/domains` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:24:30.642630 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50958df3-3952-4902-b47a-c58e94d20f2a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:24:30.642941 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50958df3-3952-4902-b47a-c58e94d20f2a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:24:30.644363 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50958df3-3952-4902-b47a-c58e94d20f2a None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:24:30.670997 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 350/692] 10.222.0.33 () {56 vars in 1109 bytes} [Mon Jul 25 05:24:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:24:30.672461 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54313ffd-74ea-4998-ad8a-03aa7d3eb544 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:24:30.766160 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30e858e-23c9-4632-b512-8cb203aa8a92 None None] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:24:30.766493 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30e858e-23c9-4632-b512-8cb203aa8a92 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:24:30.766719 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30e858e-23c9-4632-b512-8cb203aa8a92 None None] PATH_INFO: `/` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:24:30.769596 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 351/693] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jul 25 05:24:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:24:30.774035 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09c0f8c-b282-49be-b5f9-2d956fa12d59 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:24:30.774335 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09c0f8c-b282-49be-b5f9-2d956fa12d59 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:24:30.774610 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09c0f8c-b282-49be-b5f9-2d956fa12d59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:24:30.793333 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-c09c0f8c-b282-49be-b5f9-2d956fa12d59 None None] MFA Rules not processed for user `1eae540242d94c10b0ad33960a855f08`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:24:30.794574 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-c09c0f8c-b282-49be-b5f9-2d956fa12d59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:24:30.826283 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:24:30.850599 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:24:30.858289 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54313ffd-74ea-4998-ad8a-03aa7d3eb544 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:24:30.884973 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 352/694] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jul 25 05:24:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 25 05:24:30.899687 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-725e013d-902a-4b70-bf8b-bef5f8b66319 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:24:30.900756 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725e013d-902a-4b70-bf8b-bef5f8b66319 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:24:30.901307 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725e013d-902a-4b70-bf8b-bef5f8b66319 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:24:30.901757 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725e013d-902a-4b70-bf8b-bef5f8b66319 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:24:30.906868 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c09c0f8c-b282-49be-b5f9-2d956fa12d59 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:24:30.910394 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-725e013d-902a-4b70-bf8b-bef5f8b66319 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:24:30.912041 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-725e013d-902a-4b70-bf8b-bef5f8b66319 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:24:30.927319 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application [None req-725e013d-902a-4b70-bf8b-bef5f8b66319 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 05:24:30.927319 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:24:30.927319 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:24:30.927319 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:24:30.927319 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:24:30.927319 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:24:30.927319 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:24:30.927319 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:24:30.927319 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:24:30.927319 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:24:30.927319 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:24:30.927319 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:24:30.927319 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 25 05:24:30.927319 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 25 05:24:30.927319 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 25 05:24:30.928254 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 25 05:24:30.928254 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:24:30.928254 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:24:30.928254 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:24:30.928254 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:24:30.928254 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:24:30.928254 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:24:30.928254 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:24:30.928254 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application with Lock( Jul 25 05:24:30.928254 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:24:30.928254 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:24:30.928254 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:24:30.928254 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:24:30.928254 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:24:30.928254 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:24:30.928254 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:24:30.928254 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:24:30.929184 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 25 05:24:30.929184 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 25 05:24:30.929184 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 25 05:24:30.929184 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 25 05:24:30.929184 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 25 05:24:30.929184 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 25 05:24:30.929184 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 05:24:30.929184 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Jul 25 05:24:30.931690 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 343/695] 10.222.0.33 () {56 vars in 1111 bytes} [Mon Jul 25 05:24:30 2022] GET /identity/v3/projects/service => generated 88 bytes in 258 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:24:30.943519 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-725e013d-902a-4b70-bf8b-bef5f8b66319 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:24:31.039785 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c09c0f8c-b282-49be-b5f9-2d956fa12d59 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:24:31.070565 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-725e013d-902a-4b70-bf8b-bef5f8b66319 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:24:31.076140 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b60795-a2e1-423c-a609-5a76e62afd05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:24:31.076964 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b60795-a2e1-423c-a609-5a76e62afd05 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:24:31.077249 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b60795-a2e1-423c-a609-5a76e62afd05 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:24:31.077519 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b60795-a2e1-423c-a609-5a76e62afd05 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:24:31.088066 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1b60795-a2e1-423c-a609-5a76e62afd05 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 05:24:31.095916 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application [None req-f1b60795-a2e1-423c-a609-5a76e62afd05 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 05:24:31.095916 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:24:31.095916 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:24:31.095916 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:24:31.095916 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:24:31.095916 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:24:31.095916 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:24:31.095916 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:24:31.095916 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:24:31.095916 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:24:31.095916 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:24:31.095916 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:24:31.095916 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 05:24:31.095916 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 05:24:31.095916 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 05:24:31.095916 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 05:24:31.096900 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 05:24:31.096900 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 05:24:31.096900 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 05:24:31.096900 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 05:24:31.096900 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:24:31.096900 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:24:31.096900 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:24:31.096900 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:24:31.096900 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:24:31.096900 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:24:31.096900 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:24:31.096900 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application with Lock( Jul 25 05:24:31.096900 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:24:31.096900 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:24:31.096900 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:24:31.096900 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:24:31.097772 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:24:31.097772 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:24:31.097772 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:24:31.097772 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:24:31.097772 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 05:24:31.097772 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 05:24:31.097772 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 05:24:31.097772 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 05:24:31.097772 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 05:24:31.097772 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 05:24:31.097772 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 05:24:31.097772 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Jul 25 05:24:31.097772 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 353/696] 10.222.0.33 () {56 vars in 1096 bytes} [Mon Jul 25 05:24:30 2022] GET /identity/v3/roles/admin => generated 83 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 25 05:24:31.103535 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b60795-a2e1-423c-a609-5a76e62afd05 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:24:31.113406 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a63c28b-cb8e-451e-b6bd-72fa75d1ee3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:24:31.114068 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a63c28b-cb8e-451e-b6bd-72fa75d1ee3e None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:24:31.114367 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a63c28b-cb8e-451e-b6bd-72fa75d1ee3e None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:24:31.114626 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a63c28b-cb8e-451e-b6bd-72fa75d1ee3e None admin] PATH_INFO: `/v3/projects` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:24:31.115423 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a63c28b-cb8e-451e-b6bd-72fa75d1ee3e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:24:31.115727 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a63c28b-cb8e-451e-b6bd-72fa75d1ee3e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:24:31.117202 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a63c28b-cb8e-451e-b6bd-72fa75d1ee3e None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:24:31.130274 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 344/697] 10.222.0.33 () {56 vars in 1148 bytes} [Mon Jul 25 05:24:30 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:24:31.136291 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:24:31.138121 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a63c28b-cb8e-451e-b6bd-72fa75d1ee3e None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:24:31.164673 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:24:31.221471 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b60795-a2e1-423c-a609-5a76e62afd05 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:24:31.258121 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78bde964-c1f7-49e0-a889-5967c920c313 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:24:31.258795 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78bde964-c1f7-49e0-a889-5967c920c313 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:24:31.259074 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78bde964-c1f7-49e0-a889-5967c920c313 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:24:31.259347 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78bde964-c1f7-49e0-a889-5967c920c313 None admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:24:31.260098 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78bde964-c1f7-49e0-a889-5967c920c313 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:24:31.260454 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78bde964-c1f7-49e0-a889-5967c920c313 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:24:31.262058 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78bde964-c1f7-49e0-a889-5967c920c313 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:24:31.271200 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 354/698] 10.222.0.33 () {56 vars in 1099 bytes} [Mon Jul 25 05:24:31 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 05:24:31.273914 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a63c28b-cb8e-451e-b6bd-72fa75d1ee3e None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:24:31.276441 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78bde964-c1f7-49e0-a889-5967c920c313 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:24:31.316366 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfdbf6ea-bc3b-4ad2-ac48-6ca12f959a9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:24:31.317050 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfdbf6ea-bc3b-4ad2-ac48-6ca12f959a9d None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:24:31.317337 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfdbf6ea-bc3b-4ad2-ac48-6ca12f959a9d None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:24:31.317614 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfdbf6ea-bc3b-4ad2-ac48-6ca12f959a9d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:24:31.319032 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfdbf6ea-bc3b-4ad2-ac48-6ca12f959a9d None admin] RBAC: Adding query filter params (role.id=200e95cafbc54269a51756ddd406082f, scope.project.id=92afba266cce4d1bad4134c351e74be3, user.id=eb135a8d5d7444cfbfa336115ba2ba44) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:24:31.319351 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfdbf6ea-bc3b-4ad2-ac48-6ca12f959a9d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:24:31.320792 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfdbf6ea-bc3b-4ad2-ac48-6ca12f959a9d None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:24:31.333063 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 345/699] 10.222.0.33 () {56 vars in 1374 bytes} [Mon Jul 25 05:24:31 2022] GET /identity/v3/role_assignments?user.id=eb135a8d5d7444cfbfa336115ba2ba44&scope.project.id=92afba266cce4d1bad4134c351e74be3&role.id=200e95cafbc54269a51756ddd406082f => generated 608 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:24:31.427584 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:24:31.451613 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78bde964-c1f7-49e0-a889-5967c920c313 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:24:31.519618 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21758dac-f0bf-4274-bef6-758286a3c218 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:24:31.520427 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21758dac-f0bf-4274-bef6-758286a3c218 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:24:31.520714 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21758dac-f0bf-4274-bef6-758286a3c218 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:24:31.521028 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21758dac-f0bf-4274-bef6-758286a3c218 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:24:31.534925 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21758dac-f0bf-4274-bef6-758286a3c218 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:24:31.536370 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21758dac-f0bf-4274-bef6-758286a3c218 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:24:31.556035 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application [None req-21758dac-f0bf-4274-bef6-758286a3c218 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 05:24:31.556035 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:24:31.556035 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:24:31.556035 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:24:31.556035 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:24:31.556035 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:24:31.556035 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:24:31.556035 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:24:31.556035 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:24:31.556035 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:24:31.556035 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:24:31.556035 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:24:31.556035 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 05:24:31.556035 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 05:24:31.556035 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 05:24:31.557035 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 05:24:31.557035 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:24:31.557035 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:24:31.557035 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:24:31.557035 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:24:31.557035 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:24:31.557035 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:24:31.557035 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:24:31.557035 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application with Lock( Jul 25 05:24:31.557035 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:24:31.557035 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:24:31.557035 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:24:31.557035 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:24:31.557035 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:24:31.557035 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:24:31.557035 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:24:31.557035 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:24:31.559065 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 05:24:31.559065 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 05:24:31.559065 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 05:24:31.559065 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Jul 25 05:24:31.559065 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 355/700] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Jul 25 05:24:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 282 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 25 05:24:31.569919 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfdbf6ea-bc3b-4ad2-ac48-6ca12f959a9d None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:24:31.676389 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:24:31.696443 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfdbf6ea-bc3b-4ad2-ac48-6ca12f959a9d None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:24:31.732874 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a66a3add-8662-42aa-9ffb-92ed32d61571 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:24:31.733544 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66a3add-8662-42aa-9ffb-92ed32d61571 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:24:31.733830 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66a3add-8662-42aa-9ffb-92ed32d61571 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:24:31.734104 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66a3add-8662-42aa-9ffb-92ed32d61571 None admin] PATH_INFO: `/v3/domains` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:24:31.734830 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a66a3add-8662-42aa-9ffb-92ed32d61571 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:24:31.735136 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a66a3add-8662-42aa-9ffb-92ed32d61571 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:24:31.736491 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a66a3add-8662-42aa-9ffb-92ed32d61571 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:24:31.748647 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 346/701] 10.222.0.33 () {56 vars in 1109 bytes} [Mon Jul 25 05:24:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:24:31.754090 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21758dac-f0bf-4274-bef6-758286a3c218 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:24:31.876107 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21758dac-f0bf-4274-bef6-758286a3c218 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:24:31.910106 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c16db9a-e06f-44c7-9314-bd28dc05db87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:24:31.911269 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c16db9a-e06f-44c7-9314-bd28dc05db87 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:24:31.911731 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c16db9a-e06f-44c7-9314-bd28dc05db87 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:24:31.912184 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c16db9a-e06f-44c7-9314-bd28dc05db87 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:24:31.915258 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:24:31.921714 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c16db9a-e06f-44c7-9314-bd28dc05db87 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:24:31.923366 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c16db9a-e06f-44c7-9314-bd28dc05db87 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:24:31.938325 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application [None req-1c16db9a-e06f-44c7-9314-bd28dc05db87 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 25 05:24:31.938325 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:24:31.938325 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:24:31.938325 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:24:31.938325 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:24:31.938325 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:24:31.938325 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:24:31.938325 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:24:31.938325 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:24:31.938325 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:24:31.938325 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:24:31.938325 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:24:31.938325 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 25 05:24:31.938325 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 25 05:24:31.938325 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 25 05:24:31.938325 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 25 05:24:31.939530 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:24:31.939530 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:24:31.939530 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 25 05:24:31.939530 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 05:24:31.939530 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 25 05:24:31.939530 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 05:24:31.939530 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:24:31.939530 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:24:31.939530 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:24:31.939530 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:24:31.939530 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:24:31.939530 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application with Lock( Jul 25 05:24:31.939530 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:24:31.939530 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:24:31.939530 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:24:31.939530 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:24:31.939530 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:24:31.939530 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:24:31.940474 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:24:31.940474 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:24:31.940474 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 25 05:24:31.940474 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 25 05:24:31.940474 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 25 05:24:31.940474 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 25 05:24:31.940474 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 25 05:24:31.940474 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 25 05:24:31.940474 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 25 05:24:31.940474 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Jul 25 05:24:31.942757 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 356/702] 10.222.0.33 () {56 vars in 1093 bytes} [Mon Jul 25 05:24:31 2022] GET /identity/v3/users/nova => generated 82 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:24:31.948516 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a66a3add-8662-42aa-9ffb-92ed32d61571 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:24:32.072690 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a66a3add-8662-42aa-9ffb-92ed32d61571 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:24:32.108179 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3593a47-8d09-4c48-9f10-a57e85b09ec6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:24:32.109125 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3593a47-8d09-4c48-9f10-a57e85b09ec6 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:24:32.109554 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3593a47-8d09-4c48-9f10-a57e85b09ec6 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:24:32.109975 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3593a47-8d09-4c48-9f10-a57e85b09ec6 None admin] PATH_INFO: `/v3/users` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:24:32.110989 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3593a47-8d09-4c48-9f10-a57e85b09ec6 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:24:32.111442 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3593a47-8d09-4c48-9f10-a57e85b09ec6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:24:32.113108 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3593a47-8d09-4c48-9f10-a57e85b09ec6 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:24:32.116927 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc494095-47ca-487a-bd21-c98ab5a1e03f None None] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:24:32.117383 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc494095-47ca-487a-bd21-c98ab5a1e03f None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:24:32.117824 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc494095-47ca-487a-bd21-c98ab5a1e03f None None] PATH_INFO: `/` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:24:32.119333 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 357/703] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jul 25 05:24:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 25 05:24:32.126287 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f818dd-c1da-494a-9a38-340993fd8e5d None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:24:32.126712 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f818dd-c1da-494a-9a38-340993fd8e5d None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:24:32.127096 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f818dd-c1da-494a-9a38-340993fd8e5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:24:32.143117 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:24:32.150349 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-e5f818dd-c1da-494a-9a38-340993fd8e5d None None] MFA Rules not processed for user `1eae540242d94c10b0ad33960a855f08`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:24:32.152557 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-e5f818dd-c1da-494a-9a38-340993fd8e5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:24:32.155272 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 347/704] 10.222.0.33 () {56 vars in 1133 bytes} [Mon Jul 25 05:24:31 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:24:32.162937 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3593a47-8d09-4c48-9f10-a57e85b09ec6 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:24:32.293739 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 358/705] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jul 25 05:24:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 05:24:32.326667 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3593a47-8d09-4c48-9f10-a57e85b09ec6 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:24:32.337849 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f818dd-c1da-494a-9a38-340993fd8e5d None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:24:32.393558 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7167165-54c9-4ced-afc7-40c282ad96ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:24:32.394829 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7167165-54c9-4ced-afc7-40c282ad96ba None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:24:32.396340 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7167165-54c9-4ced-afc7-40c282ad96ba None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:24:32.398902 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7167165-54c9-4ced-afc7-40c282ad96ba None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:24:32.490523 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f818dd-c1da-494a-9a38-340993fd8e5d None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:24:32.518885 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:24:32.534806 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:24:32.537175 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7167165-54c9-4ced-afc7-40c282ad96ba None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:24:32.538211 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec47264-a6df-4230-8534-e30620112ea4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:24:32.539090 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec47264-a6df-4230-8534-e30620112ea4 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:24:32.539382 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec47264-a6df-4230-8534-e30620112ea4 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:24:32.539488 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7167165-54c9-4ced-afc7-40c282ad96ba None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:24:32.539666 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec47264-a6df-4230-8534-e30620112ea4 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:24:32.540319 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ec47264-a6df-4230-8534-e30620112ea4 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:24:32.541699 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ec47264-a6df-4230-8534-e30620112ea4 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:24:32.565809 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application [None req-9ec47264-a6df-4230-8534-e30620112ea4 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 25 05:24:32.565809 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:24:32.565809 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 25 05:24:32.565809 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 25 05:24:32.565809 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 25 05:24:32.565809 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 25 05:24:32.565809 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 25 05:24:32.565809 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 25 05:24:32.565809 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 25 05:24:32.565809 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Jul 25 05:24:32.565809 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 25 05:24:32.565809 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Jul 25 05:24:32.565809 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:24:32.565809 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:24:32.565809 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:24:32.565809 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:24:32.566817 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:24:32.566817 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:24:32.566817 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:24:32.566817 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:24:32.566817 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:24:32.566817 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:24:32.566817 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:24:32.566817 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 25 05:24:32.566817 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 25 05:24:32.566817 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 25 05:24:32.566817 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 25 05:24:32.566817 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:24:32.566817 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:24:32.566817 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:24:32.566817 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:24:32.566817 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:24:32.567705 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:24:32.567705 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:24:32.567705 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application with Lock( Jul 25 05:24:32.567705 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:24:32.567705 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:24:32.567705 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:24:32.567705 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:24:32.567705 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:24:32.567705 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:24:32.567705 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:24:32.567705 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:24:32.567705 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 25 05:24:32.567705 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 25 05:24:32.567705 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 25 05:24:32.567705 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Jul 25 05:24:32.570699 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 359/706] 10.222.0.33 () {56 vars in 1129 bytes} [Mon Jul 25 05:24:32 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 25 05:24:32.576966 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec47264-a6df-4230-8534-e30620112ea4 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:24:32.729683 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 348/707] 10.222.0.33 () {58 vars in 1282 bytes} [Mon Jul 25 05:24:32 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 566 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 25 05:24:32.734623 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7167165-54c9-4ced-afc7-40c282ad96ba None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:24:32.755955 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec47264-a6df-4230-8534-e30620112ea4 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:24:32.818025 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc90d266-e872-4d96-b42b-c15fba7f509e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:24:32.818626 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc90d266-e872-4d96-b42b-c15fba7f509e None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:24:32.818909 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc90d266-e872-4d96-b42b-c15fba7f509e None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:24:32.819183 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc90d266-e872-4d96-b42b-c15fba7f509e None admin] PATH_INFO: `/v3/services` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:24:32.819894 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc90d266-e872-4d96-b42b-c15fba7f509e None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:24:32.820204 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc90d266-e872-4d96-b42b-c15fba7f509e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:24:32.822144 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc90d266-e872-4d96-b42b-c15fba7f509e None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:24:32.829308 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 360/708] 10.222.0.33 () {56 vars in 1124 bytes} [Mon Jul 25 05:24:32 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:24:32.830897 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73eac1a-9dcb-4ccf-bfb5-491a55cb1306 None None] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:24:32.831182 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73eac1a-9dcb-4ccf-bfb5-491a55cb1306 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:24:32.831462 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73eac1a-9dcb-4ccf-bfb5-491a55cb1306 None None] PATH_INFO: `/` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:24:32.833981 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 361/709] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jul 25 05:24:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:24:32.836579 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d73eac1a-9dcb-4ccf-bfb5-491a55cb1306 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:24:32.877642 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:24:32.881305 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7167165-54c9-4ced-afc7-40c282ad96ba None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:24:32.914480 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:24:32.917811 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8da755c-8fab-4362-9a3f-703f38b7de59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:24:32.919697 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8da755c-8fab-4362-9a3f-703f38b7de59 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:24:32.920132 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8da755c-8fab-4362-9a3f-703f38b7de59 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:24:32.920552 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8da755c-8fab-4362-9a3f-703f38b7de59 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:24:32.932163 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8da755c-8fab-4362-9a3f-703f38b7de59 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:24:32.934388 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8da755c-8fab-4362-9a3f-703f38b7de59 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:24:32.953922 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application [None req-f8da755c-8fab-4362-9a3f-703f38b7de59 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 05:24:32.953922 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:24:32.953922 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:24:32.953922 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:24:32.953922 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:24:32.953922 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:24:32.953922 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:24:32.953922 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:24:32.953922 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:24:32.953922 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:24:32.953922 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:24:32.953922 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:24:32.953922 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 05:24:32.953922 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 05:24:32.953922 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 05:24:32.955023 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 05:24:32.955023 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:24:32.955023 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:24:32.955023 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:24:32.955023 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:24:32.955023 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:24:32.955023 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:24:32.955023 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:24:32.955023 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application with Lock( Jul 25 05:24:32.955023 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:24:32.955023 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:24:32.955023 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:24:32.955023 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:24:32.955023 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:24:32.955023 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:24:32.955023 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:24:32.955023 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:24:32.956029 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 05:24:32.956029 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 05:24:32.956029 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 05:24:32.956029 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Jul 25 05:24:32.958029 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 349/710] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Jul 25 05:24:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:24:32.959473 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434ab0f8-0d7c-4f44-b388-9895301bfa7f None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:24:32.959954 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434ab0f8-0d7c-4f44-b388-9895301bfa7f None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:24:32.960358 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434ab0f8-0d7c-4f44-b388-9895301bfa7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:24:32.972121 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d73eac1a-9dcb-4ccf-bfb5-491a55cb1306 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:24:32.979392 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-434ab0f8-0d7c-4f44-b388-9895301bfa7f None None] MFA Rules not processed for user `1eae540242d94c10b0ad33960a855f08`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:24:32.980962 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-434ab0f8-0d7c-4f44-b388-9895301bfa7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:24:33.017508 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f85ee57-46eb-4c6d-9e44-43abca24a4fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:24:33.018182 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f85ee57-46eb-4c6d-9e44-43abca24a4fc None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:24:33.018464 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f85ee57-46eb-4c6d-9e44-43abca24a4fc None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:24:33.018741 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f85ee57-46eb-4c6d-9e44-43abca24a4fc None admin] PATH_INFO: `/v3/services` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:24:33.019457 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f85ee57-46eb-4c6d-9e44-43abca24a4fc None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:24:33.019762 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f85ee57-46eb-4c6d-9e44-43abca24a4fc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:24:33.022159 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f85ee57-46eb-4c6d-9e44-43abca24a4fc None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:24:33.029960 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 362/711] 10.222.0.33 () {56 vars in 1124 bytes} [Mon Jul 25 05:24:32 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:24:33.030122 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f85ee57-46eb-4c6d-9e44-43abca24a4fc None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:24:33.085460 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 350/712] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jul 25 05:24:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 05:24:33.121620 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-434ab0f8-0d7c-4f44-b388-9895301bfa7f None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:24:33.153194 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f85ee57-46eb-4c6d-9e44-43abca24a4fc None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:24:33.175157 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:24:33.196503 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dbc2c19-2d57-4334-8de8-972df05d299b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:24:33.197212 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbc2c19-2d57-4334-8de8-972df05d299b None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:24:33.197497 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbc2c19-2d57-4334-8de8-972df05d299b None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:24:33.197774 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbc2c19-2d57-4334-8de8-972df05d299b None admin] PATH_INFO: `/v3/domains` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:24:33.198515 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dbc2c19-2d57-4334-8de8-972df05d299b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:24:33.199558 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dbc2c19-2d57-4334-8de8-972df05d299b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:24:33.201199 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dbc2c19-2d57-4334-8de8-972df05d299b None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:24:33.215787 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 363/713] 10.222.0.33 () {56 vars in 1109 bytes} [Mon Jul 25 05:24:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:24:33.221047 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dbc2c19-2d57-4334-8de8-972df05d299b None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:24:33.269808 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:24:33.294227 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-434ab0f8-0d7c-4f44-b388-9895301bfa7f None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:24:33.334802 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07039a8b-9dfa-48e6-83ed-30f55aad7b64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:24:33.336078 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07039a8b-9dfa-48e6-83ed-30f55aad7b64 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:24:33.336537 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07039a8b-9dfa-48e6-83ed-30f55aad7b64 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:24:33.337040 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07039a8b-9dfa-48e6-83ed-30f55aad7b64 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:24:33.348362 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07039a8b-9dfa-48e6-83ed-30f55aad7b64 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 05:24:33.351523 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dbc2c19-2d57-4334-8de8-972df05d299b None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:24:33.356667 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application [None req-07039a8b-9dfa-48e6-83ed-30f55aad7b64 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 25 05:24:33.356667 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:24:33.356667 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:24:33.356667 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:24:33.356667 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:24:33.356667 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:24:33.356667 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:24:33.356667 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:24:33.356667 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:24:33.356667 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:24:33.356667 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:24:33.356667 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:24:33.356667 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 05:24:33.356667 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 05:24:33.356667 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 05:24:33.356667 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 05:24:33.357670 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 05:24:33.357670 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 05:24:33.357670 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 05:24:33.357670 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 05:24:33.357670 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:24:33.357670 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:24:33.357670 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:24:33.357670 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:24:33.357670 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:24:33.357670 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:24:33.357670 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:24:33.357670 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application with Lock( Jul 25 05:24:33.357670 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:24:33.357670 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:24:33.357670 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:24:33.357670 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:24:33.358619 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:24:33.358619 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:24:33.358619 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:24:33.358619 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:24:33.358619 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 05:24:33.358619 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 05:24:33.358619 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 05:24:33.358619 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 05:24:33.358619 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 05:24:33.358619 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 05:24:33.358619 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 25 05:24:33.358619 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Jul 25 05:24:33.361456 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 351/714] 10.222.0.33 () {56 vars in 1102 bytes} [Mon Jul 25 05:24:33 2022] GET /identity/v3/roles/service => generated 85 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 25 05:24:33.363085 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72701fd2-7a4e-4bde-b575-669aa7eaff8a None None] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:24:33.363549 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72701fd2-7a4e-4bde-b575-669aa7eaff8a None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:24:33.364001 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72701fd2-7a4e-4bde-b575-669aa7eaff8a None None] PATH_INFO: `/` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:24:33.369582 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 352/715] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jul 25 05:24:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:24:33.370331 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72701fd2-7a4e-4bde-b575-669aa7eaff8a None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:24:33.402721 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d88e8a6-43aa-4f90-b206-627c4cce582f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:24:33.404370 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d88e8a6-43aa-4f90-b206-627c4cce582f None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:24:33.404953 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d88e8a6-43aa-4f90-b206-627c4cce582f None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:24:33.405431 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d88e8a6-43aa-4f90-b206-627c4cce582f None admin] PATH_INFO: `/v3/projects/service` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:24:33.418574 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d88e8a6-43aa-4f90-b206-627c4cce582f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:24:33.420879 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d88e8a6-43aa-4f90-b206-627c4cce582f None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:24:33.439924 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application [None req-6d88e8a6-43aa-4f90-b206-627c4cce582f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 05:24:33.439924 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:24:33.439924 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:24:33.439924 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:24:33.439924 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:24:33.439924 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:24:33.439924 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:24:33.439924 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:24:33.439924 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:24:33.439924 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:24:33.439924 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:24:33.439924 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:24:33.439924 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 25 05:24:33.439924 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 25 05:24:33.439924 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 25 05:24:33.441681 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 25 05:24:33.441681 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:24:33.441681 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:24:33.441681 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:24:33.441681 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:24:33.441681 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:24:33.441681 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:24:33.441681 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:24:33.441681 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application with Lock( Jul 25 05:24:33.441681 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:24:33.441681 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:24:33.441681 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:24:33.441681 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:24:33.441681 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:24:33.441681 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:24:33.441681 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:24:33.441681 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:24:33.442835 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 25 05:24:33.442835 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 25 05:24:33.442835 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 25 05:24:33.442835 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 25 05:24:33.442835 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 25 05:24:33.442835 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 25 05:24:33.442835 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 05:24:33.442835 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Jul 25 05:24:33.445252 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 364/716] 10.222.0.33 () {56 vars in 1111 bytes} [Mon Jul 25 05:24:33 2022] GET /identity/v3/projects/service => generated 88 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 25 05:24:33.447575 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b176bddd-6c7c-4020-87fb-e50bfeb65634 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:24:33.448048 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b176bddd-6c7c-4020-87fb-e50bfeb65634 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:24:33.448482 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b176bddd-6c7c-4020-87fb-e50bfeb65634 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:24:33.469623 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-b176bddd-6c7c-4020-87fb-e50bfeb65634 None None] MFA Rules not processed for user `1eae540242d94c10b0ad33960a855f08`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:24:33.471578 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-b176bddd-6c7c-4020-87fb-e50bfeb65634 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:24:33.481040 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72701fd2-7a4e-4bde-b575-669aa7eaff8a None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:24:33.515824 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6043a8ec-84c0-4be7-b19a-529671b07aa7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:24:33.517360 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6043a8ec-84c0-4be7-b19a-529671b07aa7 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:24:33.517360 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6043a8ec-84c0-4be7-b19a-529671b07aa7 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:24:33.517562 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:24:33.518000 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6043a8ec-84c0-4be7-b19a-529671b07aa7 None admin] PATH_INFO: `/v3/roles` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:24:33.518378 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6043a8ec-84c0-4be7-b19a-529671b07aa7 None admin] RBAC: Adding query filter params (name=service) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:24:33.518909 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6043a8ec-84c0-4be7-b19a-529671b07aa7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:24:33.520374 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6043a8ec-84c0-4be7-b19a-529671b07aa7 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:24:33.534250 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 353/717] 10.222.0.33 () {56 vars in 1103 bytes} [Mon Jul 25 05:24:33 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:24:33.534250 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6043a8ec-84c0-4be7-b19a-529671b07aa7 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:24:33.541155 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:24:33.563889 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 365/718] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jul 25 05:24:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 05:24:33.565081 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b176bddd-6c7c-4020-87fb-e50bfeb65634 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:24:33.668930 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b176bddd-6c7c-4020-87fb-e50bfeb65634 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:24:33.677797 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6043a8ec-84c0-4be7-b19a-529671b07aa7 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:24:33.706351 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd50bb15-00ff-4e92-9b18-803550b180aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:24:33.707206 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd50bb15-00ff-4e92-9b18-803550b180aa None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:24:33.707602 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd50bb15-00ff-4e92-9b18-803550b180aa None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:24:33.708008 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd50bb15-00ff-4e92-9b18-803550b180aa None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:24:33.719121 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd50bb15-00ff-4e92-9b18-803550b180aa None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:24:33.720983 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd50bb15-00ff-4e92-9b18-803550b180aa None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:24:33.721726 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a4c2e1-973b-4e34-9663-44c6072e4fbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:24:33.722553 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a4c2e1-973b-4e34-9663-44c6072e4fbe None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:24:33.723144 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a4c2e1-973b-4e34-9663-44c6072e4fbe None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:24:33.723570 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a4c2e1-973b-4e34-9663-44c6072e4fbe None admin] PATH_INFO: `/v3/projects` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:24:33.724932 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97a4c2e1-973b-4e34-9663-44c6072e4fbe None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:24:33.725506 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97a4c2e1-973b-4e34-9663-44c6072e4fbe None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:24:33.727071 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97a4c2e1-973b-4e34-9663-44c6072e4fbe None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:24:33.741460 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 354/719] 10.222.0.33 () {56 vars in 1148 bytes} [Mon Jul 25 05:24:33 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:24:33.742507 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a4c2e1-973b-4e34-9663-44c6072e4fbe None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:24:33.744270 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application [None req-cd50bb15-00ff-4e92-9b18-803550b180aa None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 05:24:33.744270 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:24:33.744270 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:24:33.744270 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:24:33.744270 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:24:33.744270 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:24:33.744270 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:24:33.744270 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:24:33.744270 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:24:33.744270 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:24:33.744270 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:24:33.744270 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:24:33.744270 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 05:24:33.744270 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 05:24:33.744270 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 05:24:33.745379 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 05:24:33.745379 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:24:33.745379 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:24:33.745379 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:24:33.745379 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:24:33.745379 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:24:33.745379 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:24:33.745379 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:24:33.745379 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application with Lock( Jul 25 05:24:33.745379 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:24:33.745379 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:24:33.745379 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:24:33.745379 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:24:33.745379 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:24:33.745379 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:24:33.745379 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:24:33.745379 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:24:33.746956 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 05:24:33.746956 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 05:24:33.746956 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 05:24:33.746956 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Jul 25 05:24:33.752007 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 366/720] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Jul 25 05:24:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:24:33.753051 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd50bb15-00ff-4e92-9b18-803550b180aa None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:24:33.777512 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:24:33.825338 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:24:33.867785 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a4c2e1-973b-4e34-9663-44c6072e4fbe None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:24:33.893593 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd50bb15-00ff-4e92-9b18-803550b180aa None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:24:33.905411 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7736f0c1-8f34-4685-8ed6-126088c1a022 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:24:33.906580 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7736f0c1-8f34-4685-8ed6-126088c1a022 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:24:33.907096 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7736f0c1-8f34-4685-8ed6-126088c1a022 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:24:33.907565 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7736f0c1-8f34-4685-8ed6-126088c1a022 None admin] PATH_INFO: `/v3/services/network` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:24:33.908415 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7736f0c1-8f34-4685-8ed6-126088c1a022 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:24:33.910280 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7736f0c1-8f34-4685-8ed6-126088c1a022 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:24:33.929515 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application [None req-7736f0c1-8f34-4685-8ed6-126088c1a022 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jul 25 05:24:33.929515 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:24:33.929515 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 25 05:24:33.929515 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 25 05:24:33.929515 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 25 05:24:33.929515 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 25 05:24:33.929515 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 25 05:24:33.929515 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 25 05:24:33.929515 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 25 05:24:33.929515 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Jul 25 05:24:33.929515 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 25 05:24:33.929515 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Jul 25 05:24:33.929515 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:24:33.929515 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:24:33.929515 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:24:33.929515 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:24:33.929515 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:24:33.930608 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:24:33.930608 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:24:33.930608 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:24:33.930608 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:24:33.930608 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:24:33.930608 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:24:33.930608 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 25 05:24:33.930608 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 25 05:24:33.930608 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 25 05:24:33.930608 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 25 05:24:33.930608 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:24:33.930608 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:24:33.930608 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:24:33.930608 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:24:33.930608 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:24:33.930608 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:24:33.931476 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:24:33.931476 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application with Lock( Jul 25 05:24:33.931476 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:24:33.931476 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:24:33.931476 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:24:33.931476 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:24:33.931476 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:24:33.931476 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:24:33.931476 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:24:33.931476 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:24:33.931476 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 25 05:24:33.931476 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 25 05:24:33.931476 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 25 05:24:33.931476 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Jul 25 05:24:33.935794 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab31917-7b65-4615-b7fb-12083fc621a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:24:33.936084 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab31917-7b65-4615-b7fb-12083fc621a9 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:24:33.940292 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab31917-7b65-4615-b7fb-12083fc621a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:24:33.940292 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab31917-7b65-4615-b7fb-12083fc621a9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:24:33.940292 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bab31917-7b65-4615-b7fb-12083fc621a9 None admin] RBAC: Adding query filter params (role.id=39e5c666285a42fe8fc2c3afca6260d1, scope.project.id=92afba266cce4d1bad4134c351e74be3, user.id=1cd482a61e8a420db45bb532e5dae6e2) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:24:33.940292 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bab31917-7b65-4615-b7fb-12083fc621a9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:24:33.940292 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bab31917-7b65-4615-b7fb-12083fc621a9 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:24:33.940865 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 355/721] 10.222.0.33 () {56 vars in 1111 bytes} [Mon Jul 25 05:24:33 2022] GET /identity/v3/services/network => generated 88 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:24:33.942280 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7736f0c1-8f34-4685-8ed6-126088c1a022 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:24:33.946262 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 367/722] 10.222.0.33 () {56 vars in 1374 bytes} [Mon Jul 25 05:24:33 2022] GET /identity/v3/role_assignments?user.id=1cd482a61e8a420db45bb532e5dae6e2&scope.project.id=92afba266cce4d1bad4134c351e74be3&role.id=39e5c666285a42fe8fc2c3afca6260d1 => generated 259 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:24:33.947328 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab31917-7b65-4615-b7fb-12083fc621a9 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:24:34.095126 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:24:34.122451 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab31917-7b65-4615-b7fb-12083fc621a9 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:24:34.128256 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7736f0c1-8f34-4685-8ed6-126088c1a022 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:24:34.161059 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:24:34.171893 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c6c25b-d46e-4cd8-a4b6-66d6d1f1a91d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:24:34.172575 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c6c25b-d46e-4cd8-a4b6-66d6d1f1a91d None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:24:34.172864 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c6c25b-d46e-4cd8-a4b6-66d6d1f1a91d None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:24:34.173153 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c6c25b-d46e-4cd8-a4b6-66d6d1f1a91d None admin] PATH_INFO: `/v3/services` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:24:34.173876 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3c6c25b-d46e-4cd8-a4b6-66d6d1f1a91d None admin] RBAC: Adding query filter params (name=network) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:24:34.174191 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3c6c25b-d46e-4cd8-a4b6-66d6d1f1a91d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:24:34.174775 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6748793d-61c0-47eb-af6d-0ebde1e1a58e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:24:34.175585 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3c6c25b-d46e-4cd8-a4b6-66d6d1f1a91d None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:24:34.175886 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6748793d-61c0-47eb-af6d-0ebde1e1a58e None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:24:34.176374 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6748793d-61c0-47eb-af6d-0ebde1e1a58e None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:24:34.176871 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6748793d-61c0-47eb-af6d-0ebde1e1a58e None admin] PATH_INFO: `/v3/domains` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:24:34.177797 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6748793d-61c0-47eb-af6d-0ebde1e1a58e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:24:34.178394 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6748793d-61c0-47eb-af6d-0ebde1e1a58e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:24:34.179984 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6748793d-61c0-47eb-af6d-0ebde1e1a58e None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:24:34.184235 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 368/723] 10.222.0.33 () {56 vars in 1112 bytes} [Mon Jul 25 05:24:33 2022] GET /identity/v3/services?name=network => generated 124 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:24:34.184235 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f6fefa-2380-490e-aabc-7506ee80609c None None] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:24:34.184235 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f6fefa-2380-490e-aabc-7506ee80609c None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:24:34.184235 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f6fefa-2380-490e-aabc-7506ee80609c None None] PATH_INFO: `/` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:24:34.184235 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 369/724] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jul 25 05:24:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:24:34.187873 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f6fefa-2380-490e-aabc-7506ee80609c None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:24:34.207073 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 356/725] 10.222.0.33 () {56 vars in 1109 bytes} [Mon Jul 25 05:24:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:24:34.208796 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab8380c-3ee1-4dd0-997a-ff55eaca24cb None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:24:34.209311 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab8380c-3ee1-4dd0-997a-ff55eaca24cb None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:24:34.209789 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab8380c-3ee1-4dd0-997a-ff55eaca24cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:24:34.230661 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-dab8380c-3ee1-4dd0-997a-ff55eaca24cb None None] MFA Rules not processed for user `1eae540242d94c10b0ad33960a855f08`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:24:34.231876 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-dab8380c-3ee1-4dd0-997a-ff55eaca24cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:24:34.312860 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 357/726] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jul 25 05:24:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 05:24:34.314070 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab8380c-3ee1-4dd0-997a-ff55eaca24cb None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:24:34.343519 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f6fefa-2380-490e-aabc-7506ee80609c None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:24:34.380503 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9757c43-d5e8-45d6-a4eb-de058f179442 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:24:34.381219 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9757c43-d5e8-45d6-a4eb-de058f179442 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:24:34.381504 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9757c43-d5e8-45d6-a4eb-de058f179442 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:24:34.381780 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9757c43-d5e8-45d6-a4eb-de058f179442 None admin] PATH_INFO: `/v3/services` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:24:34.382492 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9757c43-d5e8-45d6-a4eb-de058f179442 None admin] RBAC: Adding query filter params (type=network) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:24:34.382798 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9757c43-d5e8-45d6-a4eb-de058f179442 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:24:34.384228 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9757c43-d5e8-45d6-a4eb-de058f179442 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:24:34.389370 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 370/727] 10.222.0.33 () {56 vars in 1112 bytes} [Mon Jul 25 05:24:34 2022] GET /identity/v3/services?type=network => generated 124 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 05:24:34.390177 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9757c43-d5e8-45d6-a4eb-de058f179442 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:24:34.425973 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:24:34.450437 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab8380c-3ee1-4dd0-997a-ff55eaca24cb None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:24:34.474411 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:24:34.495362 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f14865-7345-4d12-b1ad-9cf7c165bca1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:24:34.496814 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f14865-7345-4d12-b1ad-9cf7c165bca1 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:24:34.497253 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f14865-7345-4d12-b1ad-9cf7c165bca1 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:24:34.497530 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f14865-7345-4d12-b1ad-9cf7c165bca1 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:24:34.506301 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7f14865-7345-4d12-b1ad-9cf7c165bca1 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:24:34.507925 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7f14865-7345-4d12-b1ad-9cf7c165bca1 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:24:34.526249 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application [None req-e7f14865-7345-4d12-b1ad-9cf7c165bca1 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 25 05:24:34.526249 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:24:34.526249 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:24:34.526249 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:24:34.526249 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:24:34.526249 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:24:34.526249 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:24:34.526249 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:24:34.526249 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:24:34.526249 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:24:34.526249 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:24:34.526249 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:24:34.526249 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 25 05:24:34.526249 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 25 05:24:34.526249 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 25 05:24:34.526249 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 25 05:24:34.527296 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:24:34.527296 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:24:34.527296 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 25 05:24:34.527296 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 05:24:34.527296 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 25 05:24:34.527296 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 05:24:34.527296 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:24:34.527296 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:24:34.527296 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:24:34.527296 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:24:34.527296 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:24:34.527296 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application with Lock( Jul 25 05:24:34.527296 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:24:34.527296 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:24:34.527296 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:24:34.527296 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:24:34.527296 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:24:34.527296 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:24:34.528677 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:24:34.528677 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:24:34.528677 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 25 05:24:34.528677 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 25 05:24:34.528677 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 25 05:24:34.528677 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 25 05:24:34.528677 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 25 05:24:34.528677 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 25 05:24:34.528677 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 25 05:24:34.528677 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Jul 25 05:24:34.528677 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 358/728] 10.222.0.33 () {56 vars in 1117 bytes} [Mon Jul 25 05:24:34 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:24:34.537839 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f14865-7345-4d12-b1ad-9cf7c165bca1 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:24:34.608370 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9757c43-d5e8-45d6-a4eb-de058f179442 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:24:34.645959 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98261faa-0dcb-42e2-93c9-39da6189128d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:24:34.646774 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98261faa-0dcb-42e2-93c9-39da6189128d None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:24:34.647062 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98261faa-0dcb-42e2-93c9-39da6189128d None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:24:34.647397 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98261faa-0dcb-42e2-93c9-39da6189128d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:24:34.653509 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f14865-7345-4d12-b1ad-9cf7c165bca1 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:24:34.661073 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98261faa-0dcb-42e2-93c9-39da6189128d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 05:24:34.672145 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application [None req-98261faa-0dcb-42e2-93c9-39da6189128d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 05:24:34.672145 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:24:34.672145 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:24:34.672145 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:24:34.672145 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:24:34.672145 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:24:34.672145 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:24:34.672145 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:24:34.672145 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:24:34.672145 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:24:34.672145 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:24:34.672145 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:24:34.672145 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 05:24:34.672145 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 05:24:34.672145 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 05:24:34.672145 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 05:24:34.673126 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 05:24:34.673126 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 05:24:34.673126 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 05:24:34.673126 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 05:24:34.673126 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:24:34.673126 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:24:34.673126 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:24:34.673126 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:24:34.673126 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:24:34.673126 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:24:34.673126 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:24:34.673126 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application with Lock( Jul 25 05:24:34.673126 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:24:34.673126 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:24:34.673126 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:24:34.673126 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:24:34.673975 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:24:34.673975 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:24:34.673975 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:24:34.673975 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:24:34.673975 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 05:24:34.673975 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 05:24:34.673975 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 05:24:34.673975 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 05:24:34.673975 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 05:24:34.673975 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 05:24:34.673975 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 05:24:34.673975 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Jul 25 05:24:34.675315 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 371/729] 10.222.0.33 () {56 vars in 1096 bytes} [Mon Jul 25 05:24:34 2022] GET /identity/v3/roles/admin => generated 83 bytes in 284 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:24:34.679349 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98261faa-0dcb-42e2-93c9-39da6189128d None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:24:34.703847 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7819140-a18b-44cb-a016-4167e637651f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:24:34.704502 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7819140-a18b-44cb-a016-4167e637651f None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:24:34.704811 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7819140-a18b-44cb-a016-4167e637651f None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:24:34.705102 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7819140-a18b-44cb-a016-4167e637651f None admin] PATH_INFO: `/v3/users` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:24:34.706029 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7819140-a18b-44cb-a016-4167e637651f None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:24:34.706840 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7819140-a18b-44cb-a016-4167e637651f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:24:34.708613 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7819140-a18b-44cb-a016-4167e637651f None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:24:34.729141 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 359/730] 10.222.0.33 () {56 vars in 1149 bytes} [Mon Jul 25 05:24:34 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:24:34.735662 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7819140-a18b-44cb-a016-4167e637651f None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:24:34.821886 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:24:34.824556 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:24:34.858516 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98261faa-0dcb-42e2-93c9-39da6189128d None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:24:34.934387 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec1cdc2-bae0-4a69-87de-9b829d39b8c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:24:34.935069 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec1cdc2-bae0-4a69-87de-9b829d39b8c5 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:24:34.935359 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec1cdc2-bae0-4a69-87de-9b829d39b8c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:24:34.935643 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec1cdc2-bae0-4a69-87de-9b829d39b8c5 None admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:24:34.936405 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ec1cdc2-bae0-4a69-87de-9b829d39b8c5 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:24:34.936718 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ec1cdc2-bae0-4a69-87de-9b829d39b8c5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:24:34.940648 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ec1cdc2-bae0-4a69-87de-9b829d39b8c5 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:24:34.952701 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7819140-a18b-44cb-a016-4167e637651f None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:24:34.960563 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 372/731] 10.222.0.33 () {56 vars in 1099 bytes} [Mon Jul 25 05:24:34 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 281 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:24:34.966746 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec1cdc2-bae0-4a69-87de-9b829d39b8c5 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:24:35.006428 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb65e376-4ec9-4f61-8b81-f5a69dc5f253 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:24:35.007455 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb65e376-4ec9-4f61-8b81-f5a69dc5f253 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:24:35.007937 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb65e376-4ec9-4f61-8b81-f5a69dc5f253 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:24:35.008406 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb65e376-4ec9-4f61-8b81-f5a69dc5f253 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:24:35.082892 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb65e376-4ec9-4f61-8b81-f5a69dc5f253 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:24:35.084687 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb65e376-4ec9-4f61-8b81-f5a69dc5f253 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:24:35.121534 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:24:35.161398 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec1cdc2-bae0-4a69-87de-9b829d39b8c5 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:24:35.237906 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04fe0065-6b62-47f3-bd77-c1b46ad1f7fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:24:35.239073 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04fe0065-6b62-47f3-bd77-c1b46ad1f7fa None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:24:35.239555 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04fe0065-6b62-47f3-bd77-c1b46ad1f7fa None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:24:35.240026 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04fe0065-6b62-47f3-bd77-c1b46ad1f7fa None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:24:35.245078 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:24:35.264229 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04fe0065-6b62-47f3-bd77-c1b46ad1f7fa None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:24:35.265715 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04fe0065-6b62-47f3-bd77-c1b46ad1f7fa None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:24:35.270708 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 360/732] 10.222.0.33 () {58 vars in 1282 bytes} [Mon Jul 25 05:24:34 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 532 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 25 05:24:35.279648 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb65e376-4ec9-4f61-8b81-f5a69dc5f253 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:24:35.307079 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application [None req-04fe0065-6b62-47f3-bd77-c1b46ad1f7fa None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 05:24:35.307079 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:24:35.307079 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:24:35.307079 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:24:35.307079 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:24:35.307079 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:24:35.307079 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:24:35.307079 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:24:35.307079 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:24:35.307079 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:24:35.307079 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:24:35.307079 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:24:35.307079 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 05:24:35.307079 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 05:24:35.307079 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 05:24:35.308062 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 05:24:35.308062 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:24:35.308062 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:24:35.308062 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:24:35.308062 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:24:35.308062 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:24:35.308062 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:24:35.308062 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:24:35.308062 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application with Lock( Jul 25 05:24:35.308062 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:24:35.308062 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:24:35.308062 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:24:35.308062 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:24:35.308062 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:24:35.308062 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:24:35.308062 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:24:35.308062 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:24:35.308978 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 05:24:35.308978 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 05:24:35.308978 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 05:24:35.308978 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Jul 25 05:24:35.315722 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 373/733] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Jul 25 05:24:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 350 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:24:35.316581 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04fe0065-6b62-47f3-bd77-c1b46ad1f7fa None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:24:35.424259 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb65e376-4ec9-4f61-8b81-f5a69dc5f253 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:24:35.435850 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04fe0065-6b62-47f3-bd77-c1b46ad1f7fa None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:24:35.470190 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b15657-5491-4056-90ba-42e09c82175b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:24:35.471081 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b15657-5491-4056-90ba-42e09c82175b None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:24:35.471449 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b15657-5491-4056-90ba-42e09c82175b None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:24:35.471734 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b15657-5491-4056-90ba-42e09c82175b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:24:35.474059 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee6961ac-6874-4816-96e8-94c350a9392d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:24:35.474671 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:24:35.475483 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6961ac-6874-4816-96e8-94c350a9392d None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:24:35.475910 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6961ac-6874-4816-96e8-94c350a9392d None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:24:35.476263 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6961ac-6874-4816-96e8-94c350a9392d None admin] PATH_INFO: `/v3/domains` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:24:35.477113 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee6961ac-6874-4816-96e8-94c350a9392d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:24:35.477425 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee6961ac-6874-4816-96e8-94c350a9392d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:24:35.478804 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee6961ac-6874-4816-96e8-94c350a9392d None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:24:35.485249 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59b15657-5491-4056-90ba-42e09c82175b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:24:35.486756 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59b15657-5491-4056-90ba-42e09c82175b None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:24:35.493490 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 374/734] 10.222.0.33 () {56 vars in 1109 bytes} [Mon Jul 25 05:24:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:24:35.494808 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02796969-eb78-4b24-8ef6-182ea15bb5e9 None None] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:24:35.495106 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02796969-eb78-4b24-8ef6-182ea15bb5e9 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:24:35.495388 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02796969-eb78-4b24-8ef6-182ea15bb5e9 None None] PATH_INFO: `/` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:24:35.500869 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 375/735] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jul 25 05:24:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:24:35.501662 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02796969-eb78-4b24-8ef6-182ea15bb5e9 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:24:35.509128 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application [None req-59b15657-5491-4056-90ba-42e09c82175b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 05:24:35.509128 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:24:35.509128 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:24:35.509128 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:24:35.509128 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:24:35.509128 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:24:35.509128 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:24:35.509128 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:24:35.509128 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:24:35.509128 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:24:35.509128 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:24:35.509128 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:24:35.509128 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 05:24:35.509128 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 05:24:35.509128 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 05:24:35.511008 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 05:24:35.511008 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:24:35.511008 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:24:35.511008 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:24:35.511008 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:24:35.511008 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:24:35.511008 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:24:35.511008 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:24:35.511008 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application with Lock( Jul 25 05:24:35.511008 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:24:35.511008 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:24:35.511008 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:24:35.511008 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:24:35.511008 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:24:35.511008 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:24:35.511008 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:24:35.511008 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:24:35.512976 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 05:24:35.512976 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 05:24:35.512976 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 05:24:35.512976 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Jul 25 05:24:35.516819 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 361/736] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Jul 25 05:24:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 25 05:24:35.518694 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd07c119-5e9e-4de3-91c6-101198f153c9 None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:24:35.518980 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd07c119-5e9e-4de3-91c6-101198f153c9 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:24:35.519538 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd07c119-5e9e-4de3-91c6-101198f153c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:24:35.541801 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-dd07c119-5e9e-4de3-91c6-101198f153c9 None None] MFA Rules not processed for user `1eae540242d94c10b0ad33960a855f08`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:24:35.543653 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-dd07c119-5e9e-4de3-91c6-101198f153c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:24:35.558440 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:24:35.619195 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02796969-eb78-4b24-8ef6-182ea15bb5e9 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:24:35.630465 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 362/737] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jul 25 05:24:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 05:24:35.631144 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd07c119-5e9e-4de3-91c6-101198f153c9 None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:24:35.680565 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa142991-070d-496b-9474-29c95d54ff80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:24:35.681444 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa142991-070d-496b-9474-29c95d54ff80 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:24:35.681841 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa142991-070d-496b-9474-29c95d54ff80 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:24:35.682202 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa142991-070d-496b-9474-29c95d54ff80 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:24:35.690936 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa142991-070d-496b-9474-29c95d54ff80 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:24:35.692504 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa142991-070d-496b-9474-29c95d54ff80 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:24:35.709219 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application [None req-fa142991-070d-496b-9474-29c95d54ff80 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 25 05:24:35.709219 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:24:35.709219 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:24:35.709219 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:24:35.709219 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:24:35.709219 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:24:35.709219 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:24:35.709219 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:24:35.709219 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:24:35.709219 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:24:35.709219 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:24:35.709219 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:24:35.709219 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 25 05:24:35.709219 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 25 05:24:35.709219 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 25 05:24:35.709219 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 25 05:24:35.713198 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:24:35.713198 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:24:35.713198 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 25 05:24:35.713198 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 05:24:35.713198 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 25 05:24:35.713198 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 05:24:35.713198 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:24:35.713198 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:24:35.713198 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:24:35.713198 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:24:35.713198 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:24:35.713198 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application with Lock( Jul 25 05:24:35.713198 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:24:35.713198 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:24:35.713198 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:24:35.713198 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:24:35.713198 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:24:35.713198 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:24:35.716299 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:24:35.716299 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:24:35.716299 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 25 05:24:35.716299 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 25 05:24:35.716299 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 25 05:24:35.716299 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 25 05:24:35.716299 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 25 05:24:35.716299 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 25 05:24:35.716299 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 25 05:24:35.716299 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Jul 25 05:24:35.716299 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 376/738] 10.222.0.33 () {56 vars in 1096 bytes} [Mon Jul 25 05:24:35 2022] GET /identity/v3/users/swift => generated 83 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:24:35.716299 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa142991-070d-496b-9474-29c95d54ff80 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:24:35.738433 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd07c119-5e9e-4de3-91c6-101198f153c9 None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:24:35.756747 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:24:35.775148 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-218ea619-f4a9-4ed4-a384-e9fb3e3da089 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:24:35.775813 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218ea619-f4a9-4ed4-a384-e9fb3e3da089 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:24:35.776094 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218ea619-f4a9-4ed4-a384-e9fb3e3da089 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:24:35.776384 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218ea619-f4a9-4ed4-a384-e9fb3e3da089 None admin] PATH_INFO: `/v3/domains` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:24:35.777155 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-218ea619-f4a9-4ed4-a384-e9fb3e3da089 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:24:35.777466 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-218ea619-f4a9-4ed4-a384-e9fb3e3da089 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:24:35.778867 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-218ea619-f4a9-4ed4-a384-e9fb3e3da089 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:24:35.796365 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 363/739] 10.222.0.33 () {56 vars in 1109 bytes} [Mon Jul 25 05:24:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:24:35.808471 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-218ea619-f4a9-4ed4-a384-e9fb3e3da089 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:24:35.831563 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa142991-070d-496b-9474-29c95d54ff80 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:24:35.840926 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:24:35.874396 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95195822-ef8f-4961-a378-0351357c4065 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:24:35.875672 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95195822-ef8f-4961-a378-0351357c4065 None admin] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:24:35.876158 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95195822-ef8f-4961-a378-0351357c4065 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:24:35.876625 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95195822-ef8f-4961-a378-0351357c4065 None admin] PATH_INFO: `/v3/services` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:24:35.877865 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95195822-ef8f-4961-a378-0351357c4065 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:24:35.880275 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95195822-ef8f-4961-a378-0351357c4065 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:24:35.905509 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 377/740] 10.222.0.33 () {60 vars in 1141 bytes} [Mon Jul 25 05:24:35 2022] POST /identity/v3/services => generated 251 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 05:24:35.905863 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95195822-ef8f-4961-a378-0351357c4065 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:24:35.934364 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-218ea619-f4a9-4ed4-a384-e9fb3e3da089 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:24:36.026121 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-565f7de8-6219-465d-816d-8d4078c70395 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:24:36.027237 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565f7de8-6219-465d-816d-8d4078c70395 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:24:36.027542 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565f7de8-6219-465d-816d-8d4078c70395 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:24:36.027821 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565f7de8-6219-465d-816d-8d4078c70395 None admin] PATH_INFO: `/v3/users` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:24:36.028679 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-565f7de8-6219-465d-816d-8d4078c70395 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:24:36.029012 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-565f7de8-6219-465d-816d-8d4078c70395 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:24:36.030457 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-565f7de8-6219-465d-816d-8d4078c70395 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:24:36.076732 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 364/741] 10.222.0.33 () {56 vars in 1135 bytes} [Mon Jul 25 05:24:35 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 280 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:24:36.084318 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-565f7de8-6219-465d-816d-8d4078c70395 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:24:36.106218 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95195822-ef8f-4961-a378-0351357c4065 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:24:36.130610 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:24:36.148183 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfcfe3bb-3c68-4b38-b86b-0b0a65c24c4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:24:36.149801 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfcfe3bb-3c68-4b38-b86b-0b0a65c24c4c None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:24:36.149801 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfcfe3bb-3c68-4b38-b86b-0b0a65c24c4c None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:24:36.149801 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfcfe3bb-3c68-4b38-b86b-0b0a65c24c4c None admin] PATH_INFO: `/v3/projects/service` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:24:36.158510 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfcfe3bb-3c68-4b38-b86b-0b0a65c24c4c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:24:36.159024 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfcfe3bb-3c68-4b38-b86b-0b0a65c24c4c None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:24:36.177448 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application [None req-cfcfe3bb-3c68-4b38-b86b-0b0a65c24c4c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 05:24:36.177448 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:24:36.177448 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:24:36.177448 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:24:36.177448 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:24:36.177448 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:24:36.177448 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:24:36.177448 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:24:36.177448 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:24:36.177448 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:24:36.177448 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:24:36.177448 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:24:36.177448 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 25 05:24:36.177448 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 25 05:24:36.177448 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 25 05:24:36.178384 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 25 05:24:36.178384 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:24:36.178384 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:24:36.178384 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:24:36.178384 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:24:36.178384 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:24:36.178384 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:24:36.178384 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:24:36.178384 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application with Lock( Jul 25 05:24:36.178384 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:24:36.178384 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:24:36.178384 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:24:36.178384 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:24:36.178384 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:24:36.178384 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:24:36.178384 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:24:36.178384 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:24:36.179289 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 25 05:24:36.179289 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 25 05:24:36.179289 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 25 05:24:36.179289 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 25 05:24:36.179289 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 25 05:24:36.179289 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 25 05:24:36.179289 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 05:24:36.179289 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Jul 25 05:24:36.179289 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 378/742] 10.222.0.33 () {56 vars in 1111 bytes} [Mon Jul 25 05:24:35 2022] GET /identity/v3/projects/service => generated 88 bytes in 273 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:24:36.180736 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6f193d-b2c8-4323-84d3-242d683ce957 None None] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:24:36.181123 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6f193d-b2c8-4323-84d3-242d683ce957 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:24:36.181464 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6f193d-b2c8-4323-84d3-242d683ce957 None None] PATH_INFO: `/` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:24:36.182955 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 379/743] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jul 25 05:24:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:24:36.186398 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b6f193d-b2c8-4323-84d3-242d683ce957 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:24:36.221359 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:24:36.223356 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-565f7de8-6219-465d-816d-8d4078c70395 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:24:36.263207 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6286fad2-880f-46e7-9c8c-d65b9d35ecad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:24:36.263939 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6286fad2-880f-46e7-9c8c-d65b9d35ecad None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:24:36.264235 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6286fad2-880f-46e7-9c8c-d65b9d35ecad None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:24:36.264519 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6286fad2-880f-46e7-9c8c-d65b9d35ecad None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:24:36.322399 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b6f193d-b2c8-4323-84d3-242d683ce957 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:24:36.345370 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6286fad2-880f-46e7-9c8c-d65b9d35ecad None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:24:36.347604 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6286fad2-880f-46e7-9c8c-d65b9d35ecad None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:24:36.362126 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a51042a-373f-4fc4-aaea-2641ad05cc7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:24:36.362800 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a51042a-373f-4fc4-aaea-2641ad05cc7d None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:24:36.363349 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a51042a-373f-4fc4-aaea-2641ad05cc7d None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:24:36.363349 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a51042a-373f-4fc4-aaea-2641ad05cc7d None admin] PATH_INFO: `/v3/projects` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:24:36.364154 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a51042a-373f-4fc4-aaea-2641ad05cc7d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:24:36.364464 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a51042a-373f-4fc4-aaea-2641ad05cc7d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:24:36.365961 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a51042a-373f-4fc4-aaea-2641ad05cc7d None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:24:36.383490 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 380/744] 10.222.0.33 () {56 vars in 1148 bytes} [Mon Jul 25 05:24:36 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:24:36.384928 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d301e35c-99d7-47b1-acf8-da584f8d1bd0 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:24:36.385207 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d301e35c-99d7-47b1-acf8-da584f8d1bd0 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:24:36.385668 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d301e35c-99d7-47b1-acf8-da584f8d1bd0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:24:36.388716 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:24:36.404260 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-d301e35c-99d7-47b1-acf8-da584f8d1bd0 None None] MFA Rules not processed for user `1eae540242d94c10b0ad33960a855f08`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:24:36.405559 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-d301e35c-99d7-47b1-acf8-da584f8d1bd0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:24:36.469440 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 365/745] 10.222.0.33 () {58 vars in 1282 bytes} [Mon Jul 25 05:24:36 2022] GET /identity/v3/auth/tokens => generated 903 bytes in 385 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 25 05:24:36.470328 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6286fad2-880f-46e7-9c8c-d65b9d35ecad None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:24:36.491371 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 381/746] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jul 25 05:24:36 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 05:24:36.492610 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d301e35c-99d7-47b1-acf8-da584f8d1bd0 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:24:36.515324 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:24:36.591212 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6286fad2-880f-46e7-9c8c-d65b9d35ecad None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:24:36.611398 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d301e35c-99d7-47b1-acf8-da584f8d1bd0 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:24:36.631580 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:24:36.639845 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ed8d2d-813c-471f-8745-b7883143830c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:24:36.641077 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ed8d2d-813c-471f-8745-b7883143830c None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:24:36.641771 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ed8d2d-813c-471f-8745-b7883143830c None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:24:36.642205 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ed8d2d-813c-471f-8745-b7883143830c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:24:36.643887 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38ed8d2d-813c-471f-8745-b7883143830c None admin] RBAC: Adding query filter params (role.id=200e95cafbc54269a51756ddd406082f, scope.project.id=92afba266cce4d1bad4134c351e74be3, user.id=3352531be1b54f76b864383ccc2e5ec2) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:24:36.644386 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38ed8d2d-813c-471f-8745-b7883143830c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:24:36.646942 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38ed8d2d-813c-471f-8745-b7883143830c None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:24:36.650718 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4379772d-684b-4ecd-9237-a90d588d899a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:24:36.651658 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4379772d-684b-4ecd-9237-a90d588d899a None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:24:36.652098 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4379772d-684b-4ecd-9237-a90d588d899a None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:24:36.652524 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4379772d-684b-4ecd-9237-a90d588d899a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:24:36.661442 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 366/747] 10.222.0.33 () {56 vars in 1374 bytes} [Mon Jul 25 05:24:36 2022] GET /identity/v3/role_assignments?user.id=3352531be1b54f76b864383ccc2e5ec2&scope.project.id=92afba266cce4d1bad4134c351e74be3&role.id=200e95cafbc54269a51756ddd406082f => generated 259 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:24:36.661926 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ed8d2d-813c-471f-8745-b7883143830c None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:24:36.681384 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4379772d-684b-4ecd-9237-a90d588d899a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:24:36.682888 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4379772d-684b-4ecd-9237-a90d588d899a None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:24:36.709179 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application [None req-4379772d-684b-4ecd-9237-a90d588d899a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 05:24:36.709179 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:24:36.709179 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:24:36.709179 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:24:36.709179 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:24:36.709179 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:24:36.709179 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:24:36.709179 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:24:36.709179 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:24:36.709179 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:24:36.709179 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:24:36.709179 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:24:36.709179 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 05:24:36.709179 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 05:24:36.709179 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 05:24:36.710166 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 05:24:36.710166 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:24:36.710166 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:24:36.710166 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:24:36.710166 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:24:36.710166 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:24:36.710166 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:24:36.710166 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:24:36.710166 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application with Lock( Jul 25 05:24:36.710166 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:24:36.710166 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:24:36.710166 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:24:36.710166 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:24:36.710166 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:24:36.710166 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:24:36.710166 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:24:36.710166 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:24:36.711096 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 05:24:36.711096 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 05:24:36.711096 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 05:24:36.711096 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Jul 25 05:24:36.711323 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 382/748] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Jul 25 05:24:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:24:36.717733 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4379772d-684b-4ecd-9237-a90d588d899a None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:24:36.797845 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:24:36.841153 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ed8d2d-813c-471f-8745-b7883143830c None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:24:36.845263 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4379772d-684b-4ecd-9237-a90d588d899a None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:24:36.884938 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f914bae-2806-40f5-87a1-4fd027c1c603 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:24:36.885791 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f914bae-2806-40f5-87a1-4fd027c1c603 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:24:36.886746 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f914bae-2806-40f5-87a1-4fd027c1c603 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:24:36.887031 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f914bae-2806-40f5-87a1-4fd027c1c603 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:24:36.887188 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e587c417-bd60-460a-bc0a-6f0243b4f401 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:24:36.887965 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e587c417-bd60-460a-bc0a-6f0243b4f401 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:24:36.888321 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e587c417-bd60-460a-bc0a-6f0243b4f401 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:24:36.888931 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e587c417-bd60-460a-bc0a-6f0243b4f401 None admin] PATH_INFO: `/v3/domains` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:24:36.890134 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e587c417-bd60-460a-bc0a-6f0243b4f401 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:24:36.890511 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e587c417-bd60-460a-bc0a-6f0243b4f401 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:24:36.892103 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e587c417-bd60-460a-bc0a-6f0243b4f401 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:24:36.899510 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f914bae-2806-40f5-87a1-4fd027c1c603 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 05:24:36.907366 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application [None req-8f914bae-2806-40f5-87a1-4fd027c1c603 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 05:24:36.907366 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:24:36.907366 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:24:36.907366 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:24:36.907366 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:24:36.907366 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:24:36.907366 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:24:36.907366 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:24:36.907366 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:24:36.907366 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:24:36.907366 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:24:36.907366 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:24:36.907366 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 05:24:36.907366 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 05:24:36.907366 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 05:24:36.907366 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 05:24:36.908341 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 05:24:36.908341 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 05:24:36.908341 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 05:24:36.908341 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 05:24:36.908341 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:24:36.908341 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:24:36.908341 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:24:36.908341 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:24:36.908341 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:24:36.908341 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:24:36.908341 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:24:36.908341 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application with Lock( Jul 25 05:24:36.908341 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:24:36.908341 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:24:36.908341 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:24:36.908341 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:24:36.909251 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:24:36.909251 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:24:36.909251 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:24:36.909251 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:24:36.909251 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 05:24:36.909251 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 05:24:36.909251 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 05:24:36.909251 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 05:24:36.909251 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 05:24:36.909251 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 05:24:36.909251 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 05:24:36.909251 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Jul 25 05:24:36.916020 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 383/750] 10.222.0.33 () {56 vars in 1109 bytes} [Mon Jul 25 05:24:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 05:24:36.916020 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184970f0-ed09-418b-bff3-b6dcc38abcae None None] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:24:36.916020 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184970f0-ed09-418b-bff3-b6dcc38abcae None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:24:36.916552 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 367/750] 10.222.0.33 () {56 vars in 1096 bytes} [Mon Jul 25 05:24:36 2022] GET /identity/v3/roles/admin => generated 83 bytes in 247 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:24:36.916710 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184970f0-ed09-418b-bff3-b6dcc38abcae None None] PATH_INFO: `/` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:24:36.916886 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f914bae-2806-40f5-87a1-4fd027c1c603 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:24:36.917050 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 384/751] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jul 25 05:24:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:24:36.917050 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-184970f0-ed09-418b-bff3-b6dcc38abcae None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:24:36.977382 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:24:37.041848 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-184970f0-ed09-418b-bff3-b6dcc38abcae None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:24:37.044028 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f914bae-2806-40f5-87a1-4fd027c1c603 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:24:37.078275 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:24:37.081418 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e7b823-348b-4b27-9efd-e547b41b05fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:24:37.082316 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e7b823-348b-4b27-9efd-e547b41b05fd None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:24:37.082510 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e7b823-348b-4b27-9efd-e547b41b05fd None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:24:37.082792 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e7b823-348b-4b27-9efd-e547b41b05fd None admin] PATH_INFO: `/v3/projects/service` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:24:37.088511 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-160aef7d-3f57-40a3-ae32-1091acec560a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:24:37.089480 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160aef7d-3f57-40a3-ae32-1091acec560a None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:24:37.090193 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160aef7d-3f57-40a3-ae32-1091acec560a None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:24:37.090613 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160aef7d-3f57-40a3-ae32-1091acec560a None admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:24:37.090726 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85e7b823-348b-4b27-9efd-e547b41b05fd None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:24:37.091601 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-160aef7d-3f57-40a3-ae32-1091acec560a None admin] RBAC: Adding query filter params (name=admin) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:24:37.092063 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-160aef7d-3f57-40a3-ae32-1091acec560a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:24:37.092191 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85e7b823-348b-4b27-9efd-e547b41b05fd None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:24:37.093711 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-160aef7d-3f57-40a3-ae32-1091acec560a None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:24:37.103951 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 385/752] 10.222.0.33 () {56 vars in 1099 bytes} [Mon Jul 25 05:24:36 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:24:37.105932 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c8f025-ca4a-412e-9c48-745e9ba12ad5 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:24:37.106368 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c8f025-ca4a-412e-9c48-745e9ba12ad5 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:24:37.106809 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c8f025-ca4a-412e-9c48-745e9ba12ad5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:24:37.112270 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application [None req-85e7b823-348b-4b27-9efd-e547b41b05fd None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 05:24:37.112270 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:24:37.112270 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:24:37.112270 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:24:37.112270 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:24:37.112270 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:24:37.112270 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:24:37.112270 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:24:37.112270 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:24:37.112270 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:24:37.112270 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:24:37.112270 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:24:37.112270 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 25 05:24:37.112270 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 25 05:24:37.112270 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 25 05:24:37.113362 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 25 05:24:37.113362 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:24:37.113362 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:24:37.113362 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:24:37.113362 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:24:37.113362 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:24:37.113362 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:24:37.113362 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:24:37.113362 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application with Lock( Jul 25 05:24:37.113362 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:24:37.113362 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:24:37.113362 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:24:37.113362 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:24:37.113362 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:24:37.113362 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:24:37.113362 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:24:37.113362 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:24:37.114353 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 25 05:24:37.114353 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 25 05:24:37.114353 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 25 05:24:37.114353 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 25 05:24:37.114353 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 25 05:24:37.114353 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 25 05:24:37.114353 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 05:24:37.114353 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Jul 25 05:24:37.118322 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 368/753] 10.222.0.33 () {56 vars in 1111 bytes} [Mon Jul 25 05:24:36 2022] GET /identity/v3/projects/service => generated 88 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:24:37.119163 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e7b823-348b-4b27-9efd-e547b41b05fd None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:24:37.128826 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-78c8f025-ca4a-412e-9c48-745e9ba12ad5 None None] MFA Rules not processed for user `1eae540242d94c10b0ad33960a855f08`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:24:37.130241 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-78c8f025-ca4a-412e-9c48-745e9ba12ad5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:24:37.221009 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 386/754] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jul 25 05:24:37 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 05:24:37.222025 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c8f025-ca4a-412e-9c48-745e9ba12ad5 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:24:37.239867 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e7b823-348b-4b27-9efd-e547b41b05fd None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:24:37.262675 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:24:37.293458 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ccabe1-221f-4a94-8fb4-85670d59c1fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:24:37.294244 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ccabe1-221f-4a94-8fb4-85670d59c1fd None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:24:37.294523 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ccabe1-221f-4a94-8fb4-85670d59c1fd None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:24:37.294793 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ccabe1-221f-4a94-8fb4-85670d59c1fd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:24:37.312646 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31ccabe1-221f-4a94-8fb4-85670d59c1fd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:24:37.314990 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31ccabe1-221f-4a94-8fb4-85670d59c1fd None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:24:37.336243 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application [None req-31ccabe1-221f-4a94-8fb4-85670d59c1fd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 05:24:37.336243 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:24:37.336243 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:24:37.336243 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:24:37.336243 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:24:37.336243 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:24:37.336243 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:24:37.336243 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:24:37.336243 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:24:37.336243 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:24:37.336243 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:24:37.336243 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:24:37.336243 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 05:24:37.336243 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 05:24:37.336243 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 05:24:37.337427 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 05:24:37.337427 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:24:37.337427 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:24:37.337427 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:24:37.337427 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:24:37.337427 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:24:37.337427 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:24:37.337427 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:24:37.337427 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application with Lock( Jul 25 05:24:37.337427 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:24:37.337427 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:24:37.337427 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:24:37.337427 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:24:37.337427 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:24:37.337427 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:24:37.337427 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:24:37.337427 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:24:37.338416 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 05:24:37.338416 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 05:24:37.338416 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 05:24:37.338416 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Jul 25 05:24:37.341162 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 369/755] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Jul 25 05:24:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:24:37.341980 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ccabe1-221f-4a94-8fb4-85670d59c1fd None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:24:37.358608 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c8f025-ca4a-412e-9c48-745e9ba12ad5 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:24:37.387273 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:24:37.409851 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8cd3fa2-a7db-43e1-abc1-de8776196532 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:24:37.410513 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8cd3fa2-a7db-43e1-abc1-de8776196532 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:24:37.410797 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8cd3fa2-a7db-43e1-abc1-de8776196532 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:24:37.411072 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8cd3fa2-a7db-43e1-abc1-de8776196532 None admin] PATH_INFO: `/v3/projects` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:24:37.411864 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8cd3fa2-a7db-43e1-abc1-de8776196532 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:24:37.412175 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8cd3fa2-a7db-43e1-abc1-de8776196532 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:24:37.413654 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8cd3fa2-a7db-43e1-abc1-de8776196532 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:24:37.430497 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 387/756] 10.222.0.33 () {56 vars in 1148 bytes} [Mon Jul 25 05:24:37 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:24:37.430497 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8cd3fa2-a7db-43e1-abc1-de8776196532 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:24:37.471050 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ccabe1-221f-4a94-8fb4-85670d59c1fd None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:24:37.515594 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f584c9e6-2d91-4a94-9741-0d65507eb5e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:24:37.516736 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f584c9e6-2d91-4a94-9741-0d65507eb5e6 None admin] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:24:37.517595 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f584c9e6-2d91-4a94-9741-0d65507eb5e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:24:37.518338 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f584c9e6-2d91-4a94-9741-0d65507eb5e6 None admin] PATH_INFO: `/v3/services` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:24:37.519434 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f584c9e6-2d91-4a94-9741-0d65507eb5e6 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:24:37.521330 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f584c9e6-2d91-4a94-9741-0d65507eb5e6 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:24:37.536562 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8cd3fa2-a7db-43e1-abc1-de8776196532 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:24:37.540731 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 370/757] 10.222.0.33 () {60 vars in 1141 bytes} [Mon Jul 25 05:24:37 2022] POST /identity/v3/services => generated 240 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 05:24:37.541784 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f584c9e6-2d91-4a94-9741-0d65507eb5e6 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:24:37.577193 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:24:37.621588 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-221cad4a-064a-42a7-acfc-55e59073c9e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:24:37.622184 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221cad4a-064a-42a7-acfc-55e59073c9e7 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:24:37.622467 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221cad4a-064a-42a7-acfc-55e59073c9e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:24:37.622800 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221cad4a-064a-42a7-acfc-55e59073c9e7 None admin] PATH_INFO: `/v3/domains` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:24:37.623529 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-221cad4a-064a-42a7-acfc-55e59073c9e7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:24:37.623935 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-221cad4a-064a-42a7-acfc-55e59073c9e7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:24:37.625463 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-221cad4a-064a-42a7-acfc-55e59073c9e7 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:24:37.651516 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 388/758] 10.222.0.33 () {56 vars in 1109 bytes} [Mon Jul 25 05:24:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:24:37.653781 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-221cad4a-064a-42a7-acfc-55e59073c9e7 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:24:37.666254 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f584c9e6-2d91-4a94-9741-0d65507eb5e6 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:24:37.706831 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:24:37.708694 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be6a1af-1d54-4174-98dc-b100de321bfc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:24:37.709727 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be6a1af-1d54-4174-98dc-b100de321bfc None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:24:37.710218 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be6a1af-1d54-4174-98dc-b100de321bfc None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:24:37.710661 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be6a1af-1d54-4174-98dc-b100de321bfc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:24:37.711794 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8be6a1af-1d54-4174-98dc-b100de321bfc None admin] RBAC: Adding query filter params (role.id=39e5c666285a42fe8fc2c3afca6260d1, scope.project.id=92afba266cce4d1bad4134c351e74be3, user.id=eb135a8d5d7444cfbfa336115ba2ba44) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:24:37.712300 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8be6a1af-1d54-4174-98dc-b100de321bfc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:24:37.714008 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8be6a1af-1d54-4174-98dc-b100de321bfc None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:24:37.722753 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 371/759] 10.222.0.33 () {56 vars in 1374 bytes} [Mon Jul 25 05:24:37 2022] GET /identity/v3/role_assignments?user.id=eb135a8d5d7444cfbfa336115ba2ba44&scope.project.id=92afba266cce4d1bad4134c351e74be3&role.id=39e5c666285a42fe8fc2c3afca6260d1 => generated 259 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:24:37.787307 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-221cad4a-064a-42a7-acfc-55e59073c9e7 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:24:37.826623 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce94d8f-75de-4917-a254-b80e70719bd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:24:37.827492 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce94d8f-75de-4917-a254-b80e70719bd9 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:24:37.827775 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce94d8f-75de-4917-a254-b80e70719bd9 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:24:37.828052 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce94d8f-75de-4917-a254-b80e70719bd9 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:24:37.836852 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ce94d8f-75de-4917-a254-b80e70719bd9 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:24:37.838399 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ce94d8f-75de-4917-a254-b80e70719bd9 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:24:37.856447 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application [None req-3ce94d8f-75de-4917-a254-b80e70719bd9 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 25 05:24:37.856447 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:24:37.856447 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:24:37.856447 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:24:37.856447 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:24:37.856447 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:24:37.856447 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:24:37.856447 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:24:37.856447 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:24:37.856447 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:24:37.856447 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:24:37.856447 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:24:37.856447 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 25 05:24:37.856447 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 25 05:24:37.856447 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 25 05:24:37.856447 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 25 05:24:37.857518 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:24:37.857518 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:24:37.857518 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 25 05:24:37.857518 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 05:24:37.857518 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 25 05:24:37.857518 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 05:24:37.857518 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:24:37.857518 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:24:37.857518 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:24:37.857518 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:24:37.857518 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:24:37.857518 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application with Lock( Jul 25 05:24:37.857518 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:24:37.857518 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:24:37.857518 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:24:37.857518 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:24:37.857518 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:24:37.857518 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:24:37.858473 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:24:37.858473 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:24:37.858473 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 25 05:24:37.858473 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 25 05:24:37.858473 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 25 05:24:37.858473 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 25 05:24:37.858473 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 25 05:24:37.858473 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 25 05:24:37.858473 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 25 05:24:37.858473 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Jul 25 05:24:37.861843 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 389/760] 10.222.0.33 () {56 vars in 1093 bytes} [Mon Jul 25 05:24:37 2022] GET /identity/v3/users/nova => generated 82 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 25 05:24:37.864393 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be6a1af-1d54-4174-98dc-b100de321bfc None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:24:37.980145 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:24:37.989800 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be6a1af-1d54-4174-98dc-b100de321bfc None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:24:38.024920 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b424d06-8af5-45bb-8d4f-0c6f93c05001 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:24:38.025654 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b424d06-8af5-45bb-8d4f-0c6f93c05001 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:24:38.026003 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b424d06-8af5-45bb-8d4f-0c6f93c05001 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:24:38.026292 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b424d06-8af5-45bb-8d4f-0c6f93c05001 None admin] PATH_INFO: `/v3/users` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:24:38.027676 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b424d06-8af5-45bb-8d4f-0c6f93c05001 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:24:38.027989 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b424d06-8af5-45bb-8d4f-0c6f93c05001 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:24:38.029461 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b424d06-8af5-45bb-8d4f-0c6f93c05001 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:24:38.047702 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 372/761] 10.222.0.33 () {56 vars in 1133 bytes} [Mon Jul 25 05:24:37 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:24:38.053299 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce94d8f-75de-4917-a254-b80e70719bd9 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:24:38.154952 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:24:38.162268 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce94d8f-75de-4917-a254-b80e70719bd9 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:24:38.205606 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc8e75a-bf00-47ea-8e70-3a303abc15e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:24:38.206408 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc8e75a-bf00-47ea-8e70-3a303abc15e1 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:24:38.206694 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc8e75a-bf00-47ea-8e70-3a303abc15e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:24:38.206973 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc8e75a-bf00-47ea-8e70-3a303abc15e1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:24:38.224403 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bc8e75a-bf00-47ea-8e70-3a303abc15e1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:24:38.225884 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bc8e75a-bf00-47ea-8e70-3a303abc15e1 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:24:38.248325 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application [None req-8bc8e75a-bf00-47ea-8e70-3a303abc15e1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 05:24:38.248325 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:24:38.248325 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:24:38.248325 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:24:38.248325 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:24:38.248325 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:24:38.248325 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:24:38.248325 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:24:38.248325 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:24:38.248325 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:24:38.248325 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:24:38.248325 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:24:38.248325 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 05:24:38.248325 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 05:24:38.248325 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 05:24:38.249283 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 05:24:38.249283 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:24:38.249283 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:24:38.249283 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:24:38.249283 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:24:38.249283 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:24:38.249283 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:24:38.249283 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:24:38.249283 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application with Lock( Jul 25 05:24:38.249283 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:24:38.249283 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:24:38.249283 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:24:38.249283 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:24:38.249283 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:24:38.249283 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:24:38.249283 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:24:38.249283 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:24:38.250383 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 05:24:38.250383 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 05:24:38.250383 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 05:24:38.250383 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Jul 25 05:24:38.250383 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 390/762] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Jul 25 05:24:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 25 05:24:38.256804 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b424d06-8af5-45bb-8d4f-0c6f93c05001 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:24:38.261055 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d68eb59-c44e-47ce-9a92-73c2c2dffbb2 None None] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:24:38.261396 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d68eb59-c44e-47ce-9a92-73c2c2dffbb2 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:24:38.261674 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d68eb59-c44e-47ce-9a92-73c2c2dffbb2 None None] PATH_INFO: `/` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:24:38.263272 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 391/763] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jul 25 05:24:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:24:38.268204 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1e8338-12a5-442b-ad49-8c7cc9fdd79e None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:24:38.268493 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1e8338-12a5-442b-ad49-8c7cc9fdd79e None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:24:38.268773 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1e8338-12a5-442b-ad49-8c7cc9fdd79e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:24:38.301822 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-aa1e8338-12a5-442b-ad49-8c7cc9fdd79e None None] MFA Rules not processed for user `1eae540242d94c10b0ad33960a855f08`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:24:38.303046 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-aa1e8338-12a5-442b-ad49-8c7cc9fdd79e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:24:38.367563 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b424d06-8af5-45bb-8d4f-0c6f93c05001 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:24:38.403807 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-937cca44-5c1b-44f7-a059-c6935a935fce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:24:38.404641 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937cca44-5c1b-44f7-a059-c6935a935fce None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:24:38.404963 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937cca44-5c1b-44f7-a059-c6935a935fce None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:24:38.405239 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937cca44-5c1b-44f7-a059-c6935a935fce None admin] PATH_INFO: `/v3/domains` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:24:38.405975 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-937cca44-5c1b-44f7-a059-c6935a935fce None admin] RBAC: Adding query filter params (name=Default) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:24:38.406282 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-937cca44-5c1b-44f7-a059-c6935a935fce None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:24:38.407730 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-937cca44-5c1b-44f7-a059-c6935a935fce None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:24:38.420939 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 373/764] 10.222.0.33 () {56 vars in 1109 bytes} [Mon Jul 25 05:24:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:24:38.426950 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-937cca44-5c1b-44f7-a059-c6935a935fce None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:24:38.441226 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:24:38.506587 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 392/765] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jul 25 05:24:38 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 239 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 25 05:24:38.539070 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-937cca44-5c1b-44f7-a059-c6935a935fce None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:24:38.542278 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa1e8338-12a5-442b-ad49-8c7cc9fdd79e None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:24:38.580325 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db33584-07dd-4b38-9826-e9c80335ed32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:24:38.581187 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db33584-07dd-4b38-9826-e9c80335ed32 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:24:38.581480 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db33584-07dd-4b38-9826-e9c80335ed32 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:24:38.581824 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db33584-07dd-4b38-9826-e9c80335ed32 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:24:38.593319 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3db33584-07dd-4b38-9826-e9c80335ed32 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:24:38.594927 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3db33584-07dd-4b38-9826-e9c80335ed32 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:24:38.611139 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application [None req-3db33584-07dd-4b38-9826-e9c80335ed32 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 05:24:38.611139 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:24:38.611139 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:24:38.611139 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:24:38.611139 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:24:38.611139 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:24:38.611139 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:24:38.611139 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:24:38.611139 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:24:38.611139 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:24:38.611139 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:24:38.611139 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:24:38.611139 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 25 05:24:38.611139 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 25 05:24:38.611139 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 25 05:24:38.612091 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 25 05:24:38.612091 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:24:38.612091 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:24:38.612091 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:24:38.612091 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:24:38.612091 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:24:38.612091 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:24:38.612091 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:24:38.612091 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application with Lock( Jul 25 05:24:38.612091 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:24:38.612091 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:24:38.612091 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:24:38.612091 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:24:38.612091 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:24:38.612091 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:24:38.612091 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:24:38.612091 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:24:38.613200 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 25 05:24:38.613200 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 25 05:24:38.613200 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 25 05:24:38.613200 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 25 05:24:38.613200 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 25 05:24:38.613200 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 25 05:24:38.613200 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 05:24:38.613200 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Jul 25 05:24:38.613200 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 374/766] 10.222.0.33 () {56 vars in 1111 bytes} [Mon Jul 25 05:24:38 2022] GET /identity/v3/projects/service => generated 88 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 25 05:24:38.620066 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db33584-07dd-4b38-9826-e9c80335ed32 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:24:38.694102 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:24:38.722455 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa1e8338-12a5-442b-ad49-8c7cc9fdd79e None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:24:38.767372 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb56fd6-88f4-498d-83b0-dcb0d3b98e1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:24:38.768422 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb56fd6-88f4-498d-83b0-dcb0d3b98e1e None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:24:38.768898 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb56fd6-88f4-498d-83b0-dcb0d3b98e1e None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:24:38.769331 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb56fd6-88f4-498d-83b0-dcb0d3b98e1e None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:24:38.770232 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cb56fd6-88f4-498d-83b0-dcb0d3b98e1e None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:24:38.771757 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cb56fd6-88f4-498d-83b0-dcb0d3b98e1e None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:24:38.785983 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application [None req-9cb56fd6-88f4-498d-83b0-dcb0d3b98e1e None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 25 05:24:38.785983 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:24:38.785983 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 25 05:24:38.785983 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 25 05:24:38.785983 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 25 05:24:38.785983 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 25 05:24:38.785983 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 25 05:24:38.785983 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 25 05:24:38.785983 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 25 05:24:38.785983 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Jul 25 05:24:38.785983 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 25 05:24:38.785983 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Jul 25 05:24:38.785983 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:24:38.785983 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:24:38.785983 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:24:38.785983 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:24:38.787124 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:24:38.787124 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:24:38.787124 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:24:38.787124 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:24:38.787124 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:24:38.787124 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:24:38.787124 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:24:38.787124 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 25 05:24:38.787124 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 25 05:24:38.787124 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 25 05:24:38.787124 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 25 05:24:38.787124 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:24:38.787124 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:24:38.787124 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:24:38.787124 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:24:38.787124 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:24:38.788168 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:24:38.788168 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:24:38.788168 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application with Lock( Jul 25 05:24:38.788168 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:24:38.788168 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:24:38.788168 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:24:38.788168 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:24:38.788168 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:24:38.788168 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:24:38.788168 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:24:38.788168 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:24:38.788168 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 25 05:24:38.788168 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 25 05:24:38.788168 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 25 05:24:38.788168 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Jul 25 05:24:38.790408 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:24:38.792254 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 393/767] 10.222.0.33 () {56 vars in 1129 bytes} [Mon Jul 25 05:24:38 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 251 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:24:38.797514 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb56fd6-88f4-498d-83b0-dcb0d3b98e1e None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:24:38.858175 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db33584-07dd-4b38-9826-e9c80335ed32 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:24:38.947003 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef09429e-5efc-4e9c-bbaf-66e42df9755f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:24:38.947694 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef09429e-5efc-4e9c-bbaf-66e42df9755f None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:24:38.947976 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef09429e-5efc-4e9c-bbaf-66e42df9755f None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:24:38.948250 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef09429e-5efc-4e9c-bbaf-66e42df9755f None admin] PATH_INFO: `/v3/projects` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:24:38.949664 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef09429e-5efc-4e9c-bbaf-66e42df9755f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:24:38.949976 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef09429e-5efc-4e9c-bbaf-66e42df9755f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:24:38.951454 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef09429e-5efc-4e9c-bbaf-66e42df9755f None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:24:38.983766 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 375/768] 10.222.0.33 () {56 vars in 1148 bytes} [Mon Jul 25 05:24:38 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 364 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:24:38.989955 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef09429e-5efc-4e9c-bbaf-66e42df9755f None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:24:39.103178 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb56fd6-88f4-498d-83b0-dcb0d3b98e1e None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:24:39.107546 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef09429e-5efc-4e9c-bbaf-66e42df9755f None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:24:39.137019 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:24:39.144458 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-179a4bd9-437b-47d0-bc15-fe29e1a4e228 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:24:39.145640 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179a4bd9-437b-47d0-bc15-fe29e1a4e228 None admin] REQUEST_METHOD: `PUT` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:24:39.145930 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179a4bd9-437b-47d0-bc15-fe29e1a4e228 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:24:39.146201 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179a4bd9-437b-47d0-bc15-fe29e1a4e228 None admin] PATH_INFO: `/v3/projects/92afba266cce4d1bad4134c351e74be3/users/1cd482a61e8a420db45bb532e5dae6e2/roles/39e5c666285a42fe8fc2c3afca6260d1` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:24:39.148024 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e783f9-384b-4356-bab6-55ffce954376 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:24:39.148327 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-179a4bd9-437b-47d0-bc15-fe29e1a4e228 None admin] RBAC: Authorizing `identity:create_grant(project_id=92afba266cce4d1bad4134c351e74be3, user_id=1cd482a61e8a420db45bb532e5dae6e2, role_id=39e5c666285a42fe8fc2c3afca6260d1)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:24:39.149036 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e783f9-384b-4356-bab6-55ffce954376 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:24:39.149464 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e783f9-384b-4356-bab6-55ffce954376 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:24:39.149778 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-179a4bd9-437b-47d0-bc15-fe29e1a4e228 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:24:39.150062 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e783f9-384b-4356-bab6-55ffce954376 None admin] PATH_INFO: `/v3/services` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:24:39.150925 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9e783f9-384b-4356-bab6-55ffce954376 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:24:39.151379 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9e783f9-384b-4356-bab6-55ffce954376 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:24:39.153790 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9e783f9-384b-4356-bab6-55ffce954376 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:24:39.159554 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 394/769] 10.222.0.33 () {56 vars in 1124 bytes} [Mon Jul 25 05:24:38 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 363 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:24:39.161125 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05327a7-fb60-4126-92ab-45a87d1be13d None None] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:24:39.161666 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05327a7-fb60-4126-92ab-45a87d1be13d None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:24:39.162146 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05327a7-fb60-4126-92ab-45a87d1be13d None None] PATH_INFO: `/` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:24:39.164049 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 395/770] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jul 25 05:24:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:24:39.164963 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c05327a7-fb60-4126-92ab-45a87d1be13d None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:24:39.172253 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 376/771] 10.222.0.33 () {58 vars in 1439 bytes} [Mon Jul 25 05:24:38 2022] PUT /identity/v3/projects/92afba266cce4d1bad4134c351e74be3/users/1cd482a61e8a420db45bb532e5dae6e2/roles/39e5c666285a42fe8fc2c3afca6260d1 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 25 05:24:39.173619 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ec29e0-9f4d-4524-b439-e747566530ad None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:24:39.173911 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ec29e0-9f4d-4524-b439-e747566530ad None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:24:39.174193 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ec29e0-9f4d-4524-b439-e747566530ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:24:39.178102 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:24:39.200172 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-57ec29e0-9f4d-4524-b439-e747566530ad None None] MFA Rules not processed for user `1eae540242d94c10b0ad33960a855f08`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:24:39.202070 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-57ec29e0-9f4d-4524-b439-e747566530ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:24:39.296728 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 377/772] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jul 25 05:24:39 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 25 05:24:39.330948 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ec29e0-9f4d-4524-b439-e747566530ad None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:24:39.342645 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c05327a7-fb60-4126-92ab-45a87d1be13d None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:24:39.398415 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea8fa7d8-4417-443d-9e88-7bcc2af74cb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:24:39.399003 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8fa7d8-4417-443d-9e88-7bcc2af74cb0 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:24:39.399289 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8fa7d8-4417-443d-9e88-7bcc2af74cb0 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:24:39.399565 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8fa7d8-4417-443d-9e88-7bcc2af74cb0 None admin] PATH_INFO: `/v3/services` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:24:39.400297 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea8fa7d8-4417-443d-9e88-7bcc2af74cb0 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:24:39.401154 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea8fa7d8-4417-443d-9e88-7bcc2af74cb0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:24:39.402608 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea8fa7d8-4417-443d-9e88-7bcc2af74cb0 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:24:39.409309 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 396/773] 10.222.0.33 () {56 vars in 1124 bytes} [Mon Jul 25 05:24:39 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:24:39.415825 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea8fa7d8-4417-443d-9e88-7bcc2af74cb0 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:24:39.483489 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ec29e0-9f4d-4524-b439-e747566530ad None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:24:39.502712 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:24:39.511775 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:24:39.530700 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31612079-f62c-4b65-a840-59a9a01105ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:24:39.531652 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31612079-f62c-4b65-a840-59a9a01105ef None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:24:39.531941 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31612079-f62c-4b65-a840-59a9a01105ef None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:24:39.532518 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31612079-f62c-4b65-a840-59a9a01105ef None admin] PATH_INFO: `/v3/roles/service` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:24:39.543743 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31612079-f62c-4b65-a840-59a9a01105ef None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 05:24:39.553027 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application [None req-31612079-f62c-4b65-a840-59a9a01105ef None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 25 05:24:39.553027 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:24:39.553027 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:24:39.553027 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:24:39.553027 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:24:39.553027 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:24:39.553027 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:24:39.553027 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:24:39.553027 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:24:39.553027 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:24:39.553027 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:24:39.553027 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:24:39.553027 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 05:24:39.553027 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 05:24:39.553027 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 05:24:39.553027 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 05:24:39.554203 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 05:24:39.554203 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 05:24:39.554203 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 05:24:39.554203 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 05:24:39.554203 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:24:39.554203 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:24:39.554203 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:24:39.554203 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:24:39.554203 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:24:39.554203 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:24:39.554203 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:24:39.554203 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application with Lock( Jul 25 05:24:39.554203 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:24:39.554203 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:24:39.554203 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:24:39.554203 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:24:39.555435 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:24:39.555435 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:24:39.555435 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:24:39.555435 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:24:39.555435 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 05:24:39.555435 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 05:24:39.555435 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 05:24:39.555435 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 05:24:39.555435 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 05:24:39.555435 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 05:24:39.555435 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 25 05:24:39.555435 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Jul 25 05:24:39.555435 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 378/774] 10.222.0.33 () {56 vars in 1102 bytes} [Mon Jul 25 05:24:39 2022] GET /identity/v3/roles/service => generated 85 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 25 05:24:39.561346 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31612079-f62c-4b65-a840-59a9a01105ef None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:24:39.562527 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea8fa7d8-4417-443d-9e88-7bcc2af74cb0 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:24:39.631198 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab9bb18-e0fa-41f0-8fb3-81a6df3f0d85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:24:39.631862 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab9bb18-e0fa-41f0-8fb3-81a6df3f0d85 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:24:39.633395 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab9bb18-e0fa-41f0-8fb3-81a6df3f0d85 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:24:39.633691 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab9bb18-e0fa-41f0-8fb3-81a6df3f0d85 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:24:39.634507 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ab9bb18-e0fa-41f0-8fb3-81a6df3f0d85 None admin] RBAC: Adding query filter params (interface=public, service_id=82bb953c04a04e1e8de3fa47d88edb07, region_id=RegionOne) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:24:39.634820 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ab9bb18-e0fa-41f0-8fb3-81a6df3f0d85 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:24:39.636827 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ab9bb18-e0fa-41f0-8fb3-81a6df3f0d85 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:24:39.656206 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 397/775] 10.222.0.33 () {56 vars in 1251 bytes} [Mon Jul 25 05:24:39 2022] GET /identity/v3/endpoints?service_id=82bb953c04a04e1e8de3fa47d88edb07&interface=public®ion_id=RegionOne => generated 194 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:24:39.661752 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab9bb18-e0fa-41f0-8fb3-81a6df3f0d85 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:24:39.715168 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31612079-f62c-4b65-a840-59a9a01105ef None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:24:39.825719 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf2b7c9-85f3-436c-a20d-54e0f816f63f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:24:39.826469 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf2b7c9-85f3-436c-a20d-54e0f816f63f None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:24:39.826690 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf2b7c9-85f3-436c-a20d-54e0f816f63f None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:24:39.826969 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf2b7c9-85f3-436c-a20d-54e0f816f63f None admin] PATH_INFO: `/v3/roles` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:24:39.827734 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf2b7c9-85f3-436c-a20d-54e0f816f63f None admin] RBAC: Adding query filter params (name=service) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:24:39.828044 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf2b7c9-85f3-436c-a20d-54e0f816f63f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:24:39.829451 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf2b7c9-85f3-436c-a20d-54e0f816f63f None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:24:39.831232 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab9bb18-e0fa-41f0-8fb3-81a6df3f0d85 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:24:39.859110 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 379/776] 10.222.0.33 () {56 vars in 1103 bytes} [Mon Jul 25 05:24:39 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 298 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:24:39.864614 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf2b7c9-85f3-436c-a20d-54e0f816f63f None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:24:39.876227 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a600dad-84db-4b7a-9cd5-604e32c5dd31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:24:39.876949 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a600dad-84db-4b7a-9cd5-604e32c5dd31 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:24:39.877233 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a600dad-84db-4b7a-9cd5-604e32c5dd31 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:24:39.877508 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a600dad-84db-4b7a-9cd5-604e32c5dd31 None admin] PATH_INFO: `/v3/services` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:24:39.878197 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a600dad-84db-4b7a-9cd5-604e32c5dd31 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:24:39.879563 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a600dad-84db-4b7a-9cd5-604e32c5dd31 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:24:39.883354 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:24:39.885908 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 398/777] 10.222.0.33 () {56 vars in 1087 bytes} [Mon Jul 25 05:24:39 2022] GET /identity/v3/services => generated 772 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:24:39.973572 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:24:40.050703 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf2b7c9-85f3-436c-a20d-54e0f816f63f None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:24:40.091356 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-778da7c7-9c15-4253-8169-b636c2e85b3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:24:40.092139 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778da7c7-9c15-4253-8169-b636c2e85b3a None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:24:40.092427 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778da7c7-9c15-4253-8169-b636c2e85b3a None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:24:40.092702 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778da7c7-9c15-4253-8169-b636c2e85b3a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:24:40.105782 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-778da7c7-9c15-4253-8169-b636c2e85b3a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:24:40.107234 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-778da7c7-9c15-4253-8169-b636c2e85b3a None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:24:40.124706 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application [None req-778da7c7-9c15-4253-8169-b636c2e85b3a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 05:24:40.124706 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:24:40.124706 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:24:40.124706 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:24:40.124706 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:24:40.124706 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:24:40.124706 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:24:40.124706 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:24:40.124706 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:24:40.124706 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:24:40.124706 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:24:40.124706 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:24:40.124706 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 05:24:40.124706 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 05:24:40.124706 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 05:24:40.125684 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 05:24:40.125684 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:24:40.125684 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:24:40.125684 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:24:40.125684 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:24:40.125684 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:24:40.125684 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:24:40.125684 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:24:40.125684 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application with Lock( Jul 25 05:24:40.125684 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:24:40.125684 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:24:40.125684 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:24:40.125684 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:24:40.125684 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:24:40.125684 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:24:40.125684 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:24:40.125684 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:24:40.126793 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 05:24:40.126793 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 05:24:40.126793 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 05:24:40.126793 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Jul 25 05:24:40.126793 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 380/778] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Jul 25 05:24:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 262 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:24:40.131691 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a600dad-84db-4b7a-9cd5-604e32c5dd31 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:24:40.203960 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17ae8ea-eb6c-4365-9e9d-2f4e562e3baa None None] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:24:40.204251 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17ae8ea-eb6c-4365-9e9d-2f4e562e3baa None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:24:40.204537 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17ae8ea-eb6c-4365-9e9d-2f4e562e3baa None None] PATH_INFO: `/` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:24:40.206203 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 381/779] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jul 25 05:24:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:24:40.211267 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a0273e-24c4-416f-ac0a-441bfa76f0de None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:24:40.211557 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a0273e-24c4-416f-ac0a-441bfa76f0de None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:24:40.211841 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a0273e-24c4-416f-ac0a-441bfa76f0de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:24:40.237392 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-42a0273e-24c4-416f-ac0a-441bfa76f0de None None] MFA Rules not processed for user `1eae540242d94c10b0ad33960a855f08`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:24:40.238526 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-42a0273e-24c4-416f-ac0a-441bfa76f0de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:24:40.274328 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a600dad-84db-4b7a-9cd5-604e32c5dd31 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:24:40.336930 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5941f38c-f8c6-47e6-a59f-6d8d3eecbf62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:24:40.337859 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5941f38c-f8c6-47e6-a59f-6d8d3eecbf62 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:24:40.338154 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5941f38c-f8c6-47e6-a59f-6d8d3eecbf62 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:24:40.338848 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5941f38c-f8c6-47e6-a59f-6d8d3eecbf62 None admin] PATH_INFO: `/v3/domains` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:24:40.340478 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5941f38c-f8c6-47e6-a59f-6d8d3eecbf62 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:24:40.340809 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5941f38c-f8c6-47e6-a59f-6d8d3eecbf62 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:24:40.342189 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5941f38c-f8c6-47e6-a59f-6d8d3eecbf62 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:24:40.343950 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 382/780] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jul 25 05:24:40 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 25 05:24:40.345408 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad5f9c8-9d1c-4f54-9792-49f8ad55b56d None None] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:24:40.345898 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad5f9c8-9d1c-4f54-9792-49f8ad55b56d None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:24:40.346305 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad5f9c8-9d1c-4f54-9792-49f8ad55b56d None None] PATH_INFO: `/` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:24:40.347901 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 383/781] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jul 25 05:24:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:24:40.353037 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2a7e0e-f513-45f2-b677-1794f7af42bb None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:24:40.353430 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2a7e0e-f513-45f2-b677-1794f7af42bb None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:24:40.353809 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2a7e0e-f513-45f2-b677-1794f7af42bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:24:40.364040 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 399/782] 10.222.0.33 () {56 vars in 1109 bytes} [Mon Jul 25 05:24:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:24:40.369876 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5941f38c-f8c6-47e6-a59f-6d8d3eecbf62 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:24:40.375485 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-be2a7e0e-f513-45f2-b677-1794f7af42bb None None] MFA Rules not processed for user `1eae540242d94c10b0ad33960a855f08`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:24:40.376830 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-be2a7e0e-f513-45f2-b677-1794f7af42bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:24:40.405310 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:24:40.456750 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:24:40.476896 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 384/783] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jul 25 05:24:40 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 25 05:24:40.478029 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be2a7e0e-f513-45f2-b677-1794f7af42bb None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:24:40.512915 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5941f38c-f8c6-47e6-a59f-6d8d3eecbf62 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:24:40.550951 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4460f10-154d-4458-a336-0559658ccc5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:24:40.551870 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4460f10-154d-4458-a336-0559658ccc5f None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:24:40.552259 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4460f10-154d-4458-a336-0559658ccc5f None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:24:40.552606 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4460f10-154d-4458-a336-0559658ccc5f None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:24:40.561349 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4460f10-154d-4458-a336-0559658ccc5f None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:24:40.562875 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4460f10-154d-4458-a336-0559658ccc5f None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:24:40.580027 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application [None req-a4460f10-154d-4458-a336-0559658ccc5f None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 25 05:24:40.580027 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:24:40.580027 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:24:40.580027 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:24:40.580027 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:24:40.580027 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:24:40.580027 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:24:40.580027 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:24:40.580027 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:24:40.580027 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:24:40.580027 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:24:40.580027 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:24:40.580027 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 25 05:24:40.580027 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 25 05:24:40.580027 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 25 05:24:40.580027 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 25 05:24:40.581058 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:24:40.581058 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:24:40.581058 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 25 05:24:40.581058 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 05:24:40.581058 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 25 05:24:40.581058 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 05:24:40.581058 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:24:40.581058 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:24:40.581058 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:24:40.581058 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:24:40.581058 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:24:40.581058 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application with Lock( Jul 25 05:24:40.581058 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:24:40.581058 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:24:40.581058 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:24:40.581058 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:24:40.581058 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:24:40.581058 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:24:40.582020 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:24:40.582020 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:24:40.582020 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 25 05:24:40.582020 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 25 05:24:40.582020 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 25 05:24:40.582020 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 25 05:24:40.582020 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 25 05:24:40.582020 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 25 05:24:40.582020 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 25 05:24:40.582020 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Jul 25 05:24:40.582020 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 400/784] 10.222.0.33 () {56 vars in 1117 bytes} [Mon Jul 25 05:24:40 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:24:40.582612 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4460f10-154d-4458-a336-0559658ccc5f None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:24:40.617683 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be2a7e0e-f513-45f2-b677-1794f7af42bb None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:24:40.665368 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6960ef3b-4221-4c97-9616-b88b8c8a49ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:24:40.666503 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6960ef3b-4221-4c97-9616-b88b8c8a49ee None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:24:40.666976 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6960ef3b-4221-4c97-9616-b88b8c8a49ee None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:24:40.668592 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6960ef3b-4221-4c97-9616-b88b8c8a49ee None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:24:40.682725 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6960ef3b-4221-4c97-9616-b88b8c8a49ee None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 05:24:40.694165 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application [None req-6960ef3b-4221-4c97-9616-b88b8c8a49ee None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 05:24:40.694165 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:24:40.694165 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:24:40.694165 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:24:40.694165 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:24:40.694165 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:24:40.694165 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:24:40.694165 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:24:40.694165 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:24:40.694165 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:24:40.694165 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:24:40.694165 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:24:40.694165 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 05:24:40.694165 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 05:24:40.694165 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 05:24:40.694165 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 05:24:40.695143 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 05:24:40.695143 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 05:24:40.695143 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 05:24:40.695143 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 05:24:40.695143 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:24:40.695143 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:24:40.695143 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:24:40.695143 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:24:40.695143 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:24:40.695143 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:24:40.695143 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:24:40.695143 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application with Lock( Jul 25 05:24:40.695143 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:24:40.695143 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:24:40.695143 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:24:40.695143 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:24:40.695996 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:24:40.695996 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:24:40.695996 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:24:40.695996 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:24:40.695996 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 05:24:40.695996 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 05:24:40.695996 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 05:24:40.695996 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 05:24:40.695996 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 05:24:40.695996 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 05:24:40.695996 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 05:24:40.695996 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Jul 25 05:24:40.696667 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4460f10-154d-4458-a336-0559658ccc5f None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:24:40.703632 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 385/785] 10.222.0.33 () {56 vars in 1096 bytes} [Mon Jul 25 05:24:40 2022] GET /identity/v3/roles/admin => generated 83 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:24:40.703632 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6960ef3b-4221-4c97-9616-b88b8c8a49ee None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:24:40.720134 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:24:40.730959 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:24:40.746626 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd34c3ae-f902-41db-a32e-75a398578ae6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:24:40.747453 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd34c3ae-f902-41db-a32e-75a398578ae6 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:24:40.747737 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd34c3ae-f902-41db-a32e-75a398578ae6 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:24:40.748012 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd34c3ae-f902-41db-a32e-75a398578ae6 None admin] PATH_INFO: `/v3/services/network` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:24:40.748664 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd34c3ae-f902-41db-a32e-75a398578ae6 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:24:40.750047 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd34c3ae-f902-41db-a32e-75a398578ae6 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:24:40.765707 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application [None req-bd34c3ae-f902-41db-a32e-75a398578ae6 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jul 25 05:24:40.765707 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:24:40.765707 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 25 05:24:40.765707 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 25 05:24:40.765707 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 25 05:24:40.765707 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 25 05:24:40.765707 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 25 05:24:40.765707 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 25 05:24:40.765707 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 25 05:24:40.765707 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Jul 25 05:24:40.765707 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 25 05:24:40.765707 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Jul 25 05:24:40.765707 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:24:40.765707 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:24:40.765707 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:24:40.765707 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:24:40.765707 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:24:40.766720 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:24:40.766720 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:24:40.766720 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:24:40.766720 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:24:40.766720 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:24:40.766720 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:24:40.766720 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 25 05:24:40.766720 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 25 05:24:40.766720 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 25 05:24:40.766720 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 25 05:24:40.766720 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:24:40.766720 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:24:40.766720 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:24:40.766720 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:24:40.766720 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:24:40.766720 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:24:40.767617 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:24:40.767617 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application with Lock( Jul 25 05:24:40.767617 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:24:40.767617 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:24:40.767617 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:24:40.767617 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:24:40.767617 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:24:40.767617 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:24:40.767617 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:24:40.767617 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:24:40.767617 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 25 05:24:40.767617 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 25 05:24:40.767617 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 25 05:24:40.767617 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Jul 25 05:24:40.772116 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 401/786] 10.222.0.33 () {56 vars in 1111 bytes} [Mon Jul 25 05:24:40 2022] GET /identity/v3/services/network => generated 88 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:24:40.773172 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd34c3ae-f902-41db-a32e-75a398578ae6 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:24:40.856483 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6960ef3b-4221-4c97-9616-b88b8c8a49ee None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:24:40.900493 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d406fd-fa40-4cfb-ab27-63941bde3735 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:24:40.901459 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d406fd-fa40-4cfb-ab27-63941bde3735 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:24:40.902453 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d406fd-fa40-4cfb-ab27-63941bde3735 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:24:40.902883 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d406fd-fa40-4cfb-ab27-63941bde3735 None admin] PATH_INFO: `/v3/users` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:24:40.904010 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17d406fd-fa40-4cfb-ab27-63941bde3735 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:24:40.904567 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17d406fd-fa40-4cfb-ab27-63941bde3735 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:24:40.906173 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17d406fd-fa40-4cfb-ab27-63941bde3735 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:24:40.907347 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd34c3ae-f902-41db-a32e-75a398578ae6 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:24:40.925984 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 386/787] 10.222.0.33 () {56 vars in 1149 bytes} [Mon Jul 25 05:24:40 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:24:40.927094 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d406fd-fa40-4cfb-ab27-63941bde3735 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:24:40.944095 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d75c10-1e1c-49a4-8abc-ea1a27c015ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:24:40.944985 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d75c10-1e1c-49a4-8abc-ea1a27c015ea None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:24:40.945379 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d75c10-1e1c-49a4-8abc-ea1a27c015ea None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:24:40.945845 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d75c10-1e1c-49a4-8abc-ea1a27c015ea None admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:24:40.946708 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0d75c10-1e1c-49a4-8abc-ea1a27c015ea None admin] RBAC: Adding query filter params (name=admin) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:24:40.947107 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0d75c10-1e1c-49a4-8abc-ea1a27c015ea None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:24:40.949139 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0d75c10-1e1c-49a4-8abc-ea1a27c015ea None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:24:40.957857 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 402/788] 10.222.0.33 () {56 vars in 1099 bytes} [Mon Jul 25 05:24:40 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:24:40.962565 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d75c10-1e1c-49a4-8abc-ea1a27c015ea None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:24:41.009301 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:24:41.013926 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:24:41.047805 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d406fd-fa40-4cfb-ab27-63941bde3735 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:24:41.066833 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d75c10-1e1c-49a4-8abc-ea1a27c015ea None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:24:41.086896 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4259d2c-4364-4204-bf08-22fd7a58488a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:24:41.087892 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4259d2c-4364-4204-bf08-22fd7a58488a None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:24:41.088489 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4259d2c-4364-4204-bf08-22fd7a58488a None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:24:41.089554 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4259d2c-4364-4204-bf08-22fd7a58488a None admin] PATH_INFO: `/v3/services` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:24:41.090642 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4259d2c-4364-4204-bf08-22fd7a58488a None admin] RBAC: Adding query filter params (name=network) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:24:41.091160 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4259d2c-4364-4204-bf08-22fd7a58488a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:24:41.092750 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4259d2c-4364-4204-bf08-22fd7a58488a None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:24:41.098535 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 387/789] 10.222.0.33 () {56 vars in 1112 bytes} [Mon Jul 25 05:24:40 2022] GET /identity/v3/services?name=network => generated 124 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:24:41.099618 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4259d2c-4364-4204-bf08-22fd7a58488a None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:24:41.106620 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80892337-1efc-4905-8627-3ddc08e95ef5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:24:41.107488 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80892337-1efc-4905-8627-3ddc08e95ef5 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:24:41.107765 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80892337-1efc-4905-8627-3ddc08e95ef5 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:24:41.108091 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80892337-1efc-4905-8627-3ddc08e95ef5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:24:41.120127 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80892337-1efc-4905-8627-3ddc08e95ef5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:24:41.122359 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80892337-1efc-4905-8627-3ddc08e95ef5 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:24:41.140879 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application [None req-80892337-1efc-4905-8627-3ddc08e95ef5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 05:24:41.140879 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:24:41.140879 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:24:41.140879 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:24:41.140879 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:24:41.140879 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:24:41.140879 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:24:41.140879 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:24:41.140879 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:24:41.140879 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:24:41.140879 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:24:41.140879 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:24:41.140879 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 05:24:41.140879 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 05:24:41.140879 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 05:24:41.141833 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 05:24:41.141833 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:24:41.141833 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:24:41.141833 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:24:41.141833 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:24:41.141833 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:24:41.141833 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:24:41.141833 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:24:41.141833 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application with Lock( Jul 25 05:24:41.141833 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:24:41.141833 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:24:41.141833 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:24:41.141833 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:24:41.141833 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:24:41.141833 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:24:41.141833 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:24:41.141833 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:24:41.142727 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 05:24:41.142727 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 05:24:41.142727 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 05:24:41.142727 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Jul 25 05:24:41.142727 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 403/790] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Jul 25 05:24:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:24:41.143018 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80892337-1efc-4905-8627-3ddc08e95ef5 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:24:41.225315 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4259d2c-4364-4204-bf08-22fd7a58488a None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:24:41.244987 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:24:41.251771 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80892337-1efc-4905-8627-3ddc08e95ef5 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:24:41.264883 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27baeb6c-4595-42e5-830e-ecf9d14f505a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:24:41.265725 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27baeb6c-4595-42e5-830e-ecf9d14f505a None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:24:41.266011 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27baeb6c-4595-42e5-830e-ecf9d14f505a None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:24:41.266285 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27baeb6c-4595-42e5-830e-ecf9d14f505a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:24:41.269409 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:24:41.279722 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27baeb6c-4595-42e5-830e-ecf9d14f505a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:24:41.281695 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27baeb6c-4595-42e5-830e-ecf9d14f505a None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:24:41.289438 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae31c974-97fb-431d-a2a8-5de30c3770db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:24:41.290179 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae31c974-97fb-431d-a2a8-5de30c3770db None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:24:41.290527 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae31c974-97fb-431d-a2a8-5de30c3770db None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:24:41.290874 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae31c974-97fb-431d-a2a8-5de30c3770db None admin] PATH_INFO: `/v3/services` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:24:41.291842 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae31c974-97fb-431d-a2a8-5de30c3770db None admin] RBAC: Adding query filter params (type=network) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:24:41.291959 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae31c974-97fb-431d-a2a8-5de30c3770db None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:24:41.293610 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae31c974-97fb-431d-a2a8-5de30c3770db None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:24:41.301924 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application [None req-27baeb6c-4595-42e5-830e-ecf9d14f505a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 05:24:41.301924 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:24:41.301924 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:24:41.301924 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:24:41.301924 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:24:41.301924 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:24:41.301924 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:24:41.301924 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:24:41.301924 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:24:41.301924 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:24:41.301924 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:24:41.301924 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:24:41.301924 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 05:24:41.301924 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 05:24:41.301924 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 05:24:41.302994 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 05:24:41.302994 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:24:41.302994 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:24:41.302994 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:24:41.302994 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:24:41.302994 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:24:41.302994 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:24:41.302994 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:24:41.302994 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application with Lock( Jul 25 05:24:41.302994 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:24:41.302994 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:24:41.302994 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:24:41.302994 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:24:41.302994 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:24:41.302994 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:24:41.302994 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:24:41.302994 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:24:41.304043 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 05:24:41.304043 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 05:24:41.304043 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 05:24:41.304043 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Jul 25 05:24:41.304399 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 404/791] 10.222.0.33 () {56 vars in 1112 bytes} [Mon Jul 25 05:24:41 2022] GET /identity/v3/services?type=network => generated 350 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:24:41.305259 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae31c974-97fb-431d-a2a8-5de30c3770db None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:24:41.306174 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 388/792] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Jul 25 05:24:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:24:41.307260 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27baeb6c-4595-42e5-830e-ecf9d14f505a None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:24:41.412639 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae31c974-97fb-431d-a2a8-5de30c3770db None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:24:41.416578 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27baeb6c-4595-42e5-830e-ecf9d14f505a None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:24:41.453725 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa8c338a-e0c1-4f56-a8a5-ba47b6922039 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:24:41.454627 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8c338a-e0c1-4f56-a8a5-ba47b6922039 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:24:41.455079 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8c338a-e0c1-4f56-a8a5-ba47b6922039 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:24:41.455498 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8c338a-e0c1-4f56-a8a5-ba47b6922039 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:24:41.455751 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f8c2a4-709e-4f0c-908a-2ebf01de350d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:24:41.456507 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f8c2a4-709e-4f0c-908a-2ebf01de350d None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:24:41.456806 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa8c338a-e0c1-4f56-a8a5-ba47b6922039 None admin] RBAC: Adding query filter params (interface=public, service_id=d2cba4e0276246659b67fb870348c43c, region_id=RegionOne) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:24:41.457001 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f8c2a4-709e-4f0c-908a-2ebf01de350d None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:24:41.457386 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f8c2a4-709e-4f0c-908a-2ebf01de350d None admin] PATH_INFO: `/v3/domains` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:24:41.457563 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa8c338a-e0c1-4f56-a8a5-ba47b6922039 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:24:41.458286 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3f8c2a4-709e-4f0c-908a-2ebf01de350d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:24:41.458697 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3f8c2a4-709e-4f0c-908a-2ebf01de350d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:24:41.459385 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa8c338a-e0c1-4f56-a8a5-ba47b6922039 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:24:41.460213 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3f8c2a4-709e-4f0c-908a-2ebf01de350d None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:24:41.467944 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 389/793] 10.222.0.33 () {56 vars in 1251 bytes} [Mon Jul 25 05:24:41 2022] GET /identity/v3/endpoints?service_id=d2cba4e0276246659b67fb870348c43c&interface=public®ion_id=RegionOne => generated 194 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:24:41.468944 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa8c338a-e0c1-4f56-a8a5-ba47b6922039 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:24:41.482064 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 405/794] 10.222.0.33 () {56 vars in 1109 bytes} [Mon Jul 25 05:24:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:24:41.482064 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f8c2a4-709e-4f0c-908a-2ebf01de350d None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:24:41.503188 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:24:41.531225 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:24:41.583207 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa8c338a-e0c1-4f56-a8a5-ba47b6922039 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:24:41.592603 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f8c2a4-709e-4f0c-908a-2ebf01de350d None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:24:41.618504 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e84796-c743-492e-b5ca-b1c606c1fc10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:24:41.619167 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e84796-c743-492e-b5ca-b1c606c1fc10 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:24:41.619450 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e84796-c743-492e-b5ca-b1c606c1fc10 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:24:41.619882 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e84796-c743-492e-b5ca-b1c606c1fc10 None admin] PATH_INFO: `/v3/domains` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:24:41.621285 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0e84796-c743-492e-b5ca-b1c606c1fc10 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:24:41.621548 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0e84796-c743-492e-b5ca-b1c606c1fc10 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:24:41.622886 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0e84796-c743-492e-b5ca-b1c606c1fc10 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:24:41.628491 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f2bf14-946f-4325-adaf-fa265c13c8aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:24:41.629186 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f2bf14-946f-4325-adaf-fa265c13c8aa None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:24:41.629473 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f2bf14-946f-4325-adaf-fa265c13c8aa None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:24:41.629754 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f2bf14-946f-4325-adaf-fa265c13c8aa None admin] PATH_INFO: `/v3/services` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:24:41.630432 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50f2bf14-946f-4325-adaf-fa265c13c8aa None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:24:41.631779 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50f2bf14-946f-4325-adaf-fa265c13c8aa None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:24:41.634922 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 390/795] 10.222.0.33 () {56 vars in 1109 bytes} [Mon Jul 25 05:24:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:24:41.635931 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e84796-c743-492e-b5ca-b1c606c1fc10 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:24:41.640935 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 406/796] 10.222.0.33 () {56 vars in 1087 bytes} [Mon Jul 25 05:24:41 2022] GET /identity/v3/services => generated 772 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:24:41.642736 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984f4b5a-227b-469a-9dbf-45abe84ef7a9 None None] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:24:41.643025 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984f4b5a-227b-469a-9dbf-45abe84ef7a9 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:24:41.643297 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984f4b5a-227b-469a-9dbf-45abe84ef7a9 None None] PATH_INFO: `/` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:24:41.649223 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 407/797] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jul 25 05:24:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:24:41.650444 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-984f4b5a-227b-469a-9dbf-45abe84ef7a9 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:24:41.766120 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e84796-c743-492e-b5ca-b1c606c1fc10 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:24:41.783321 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:24:41.785891 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:24:41.815323 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f6f215-7f92-4c45-8eef-ab9855b68a91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:24:41.815757 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-984f4b5a-227b-469a-9dbf-45abe84ef7a9 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:24:41.816128 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f6f215-7f92-4c45-8eef-ab9855b68a91 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:24:41.816412 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f6f215-7f92-4c45-8eef-ab9855b68a91 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:24:41.816688 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f6f215-7f92-4c45-8eef-ab9855b68a91 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:24:41.824608 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12f6f215-7f92-4c45-8eef-ab9855b68a91 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:24:41.826081 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12f6f215-7f92-4c45-8eef-ab9855b68a91 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:24:41.843538 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application [None req-12f6f215-7f92-4c45-8eef-ab9855b68a91 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 05:24:41.843538 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:24:41.843538 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:24:41.843538 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:24:41.843538 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:24:41.843538 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:24:41.843538 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:24:41.843538 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:24:41.843538 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:24:41.843538 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:24:41.843538 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:24:41.843538 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:24:41.843538 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 25 05:24:41.843538 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 25 05:24:41.843538 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 25 05:24:41.844628 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 25 05:24:41.844628 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:24:41.844628 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:24:41.844628 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:24:41.844628 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:24:41.844628 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:24:41.844628 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:24:41.844628 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:24:41.844628 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application with Lock( Jul 25 05:24:41.844628 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:24:41.844628 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:24:41.844628 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:24:41.844628 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:24:41.844628 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:24:41.844628 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:24:41.844628 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:24:41.844628 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:24:41.845612 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 25 05:24:41.845612 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 25 05:24:41.845612 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 25 05:24:41.845612 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 25 05:24:41.845612 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 25 05:24:41.845612 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 25 05:24:41.845612 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 05:24:41.845612 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Jul 25 05:24:41.845612 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 391/798] 10.222.0.33 () {56 vars in 1111 bytes} [Mon Jul 25 05:24:41 2022] GET /identity/v3/projects/service => generated 88 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:24:41.846321 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc202462-dba6-4d6d-8af0-9a8104c34c1f None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:24:41.846611 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc202462-dba6-4d6d-8af0-9a8104c34c1f None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:24:41.846890 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc202462-dba6-4d6d-8af0-9a8104c34c1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:24:41.861541 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c17d4c8a-1968-46ce-898d-0d6cb616937e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:24:41.862592 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17d4c8a-1968-46ce-898d-0d6cb616937e None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:24:41.862875 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17d4c8a-1968-46ce-898d-0d6cb616937e None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:24:41.863143 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17d4c8a-1968-46ce-898d-0d6cb616937e None admin] PATH_INFO: `/v3/users/swift` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:24:41.864325 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-bc202462-dba6-4d6d-8af0-9a8104c34c1f None None] MFA Rules not processed for user `1eae540242d94c10b0ad33960a855f08`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:24:41.865548 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-bc202462-dba6-4d6d-8af0-9a8104c34c1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:24:41.874960 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c17d4c8a-1968-46ce-898d-0d6cb616937e None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:24:41.877684 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c17d4c8a-1968-46ce-898d-0d6cb616937e None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:24:41.898902 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application [None req-c17d4c8a-1968-46ce-898d-0d6cb616937e None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 25 05:24:41.898902 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:24:41.898902 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:24:41.898902 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:24:41.898902 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:24:41.898902 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:24:41.898902 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:24:41.898902 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:24:41.898902 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:24:41.898902 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:24:41.898902 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:24:41.898902 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:24:41.898902 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 25 05:24:41.898902 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 25 05:24:41.898902 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 25 05:24:41.898902 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 25 05:24:41.899980 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:24:41.899980 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:24:41.899980 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 25 05:24:41.899980 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 05:24:41.899980 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 25 05:24:41.899980 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 05:24:41.899980 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:24:41.899980 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:24:41.899980 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:24:41.899980 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:24:41.899980 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:24:41.899980 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application with Lock( Jul 25 05:24:41.899980 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:24:41.899980 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:24:41.899980 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:24:41.899980 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:24:41.899980 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:24:41.899980 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:24:41.900973 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:24:41.900973 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:24:41.900973 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 25 05:24:41.900973 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 25 05:24:41.900973 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 25 05:24:41.900973 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 25 05:24:41.900973 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 25 05:24:41.900973 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 25 05:24:41.900973 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 25 05:24:41.900973 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Jul 25 05:24:41.908095 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 408/799] 10.222.0.33 () {56 vars in 1096 bytes} [Mon Jul 25 05:24:41 2022] GET /identity/v3/users/swift => generated 83 bytes in 258 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:24:41.908095 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c17d4c8a-1968-46ce-898d-0d6cb616937e None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:24:41.946269 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 392/800] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jul 25 05:24:41 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 25 05:24:41.947532 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc202462-dba6-4d6d-8af0-9a8104c34c1f None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:24:42.045111 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c17d4c8a-1968-46ce-898d-0d6cb616937e None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:24:42.062671 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc202462-dba6-4d6d-8af0-9a8104c34c1f None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:24:42.064821 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:24:42.084634 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af5aaa6-06c4-49b1-aee7-bc70e26a7c7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:24:42.085473 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af5aaa6-06c4-49b1-aee7-bc70e26a7c7c None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:24:42.086150 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af5aaa6-06c4-49b1-aee7-bc70e26a7c7c None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:24:42.086261 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af5aaa6-06c4-49b1-aee7-bc70e26a7c7c None admin] PATH_INFO: `/v3/projects` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:24:42.087134 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5af5aaa6-06c4-49b1-aee7-bc70e26a7c7c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:24:42.087508 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5af5aaa6-06c4-49b1-aee7-bc70e26a7c7c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:24:42.089700 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5af5aaa6-06c4-49b1-aee7-bc70e26a7c7c None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:24:42.101958 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34fa8557-0499-428f-ab10-23c60c1e0575 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:24:42.102881 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34fa8557-0499-428f-ab10-23c60c1e0575 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:24:42.103355 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34fa8557-0499-428f-ab10-23c60c1e0575 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:24:42.103849 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34fa8557-0499-428f-ab10-23c60c1e0575 None admin] PATH_INFO: `/v3/users` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:24:42.105055 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34fa8557-0499-428f-ab10-23c60c1e0575 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:24:42.105739 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34fa8557-0499-428f-ab10-23c60c1e0575 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:24:42.107372 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34fa8557-0499-428f-ab10-23c60c1e0575 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:24:42.108489 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 409/801] 10.222.0.33 () {56 vars in 1148 bytes} [Mon Jul 25 05:24:41 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:24:42.109428 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af5aaa6-06c4-49b1-aee7-bc70e26a7c7c None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:24:42.113583 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:24:42.129417 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 393/802] 10.222.0.33 () {56 vars in 1135 bytes} [Mon Jul 25 05:24:41 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:24:42.130671 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34fa8557-0499-428f-ab10-23c60c1e0575 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:24:42.222311 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af5aaa6-06c4-49b1-aee7-bc70e26a7c7c None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:24:42.240006 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34fa8557-0499-428f-ab10-23c60c1e0575 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:24:42.259214 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c002fbd-c6a3-4056-8831-d6a9a72d1723 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:24:42.260036 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c002fbd-c6a3-4056-8831-d6a9a72d1723 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:24:42.260324 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c002fbd-c6a3-4056-8831-d6a9a72d1723 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:24:42.260603 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c002fbd-c6a3-4056-8831-d6a9a72d1723 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:24:42.271801 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c002fbd-c6a3-4056-8831-d6a9a72d1723 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 05:24:42.275998 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bae1219-282a-4638-9844-9ba14478d2a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:24:42.276958 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bae1219-282a-4638-9844-9ba14478d2a2 None admin] REQUEST_METHOD: `PUT` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:24:42.277244 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bae1219-282a-4638-9844-9ba14478d2a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:24:42.277519 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bae1219-282a-4638-9844-9ba14478d2a2 None admin] PATH_INFO: `/v3/projects/92afba266cce4d1bad4134c351e74be3/users/3352531be1b54f76b864383ccc2e5ec2/roles/200e95cafbc54269a51756ddd406082f` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:24:42.279544 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application [None req-3c002fbd-c6a3-4056-8831-d6a9a72d1723 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 05:24:42.279544 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:24:42.279544 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:24:42.279544 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:24:42.279544 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:24:42.279544 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:24:42.279544 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:24:42.279544 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:24:42.279544 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:24:42.279544 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:24:42.279544 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:24:42.279544 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:24:42.279544 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 05:24:42.279544 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 05:24:42.279544 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 05:24:42.279544 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 05:24:42.280449 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 05:24:42.280449 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 05:24:42.280449 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 05:24:42.280449 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 05:24:42.280449 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:24:42.280449 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:24:42.280449 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:24:42.280449 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:24:42.280449 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:24:42.280449 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:24:42.280449 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:24:42.280449 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application with Lock( Jul 25 05:24:42.280449 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:24:42.280449 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:24:42.280449 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:24:42.280449 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:24:42.281357 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:24:42.281357 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:24:42.281357 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:24:42.281357 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:24:42.281357 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 05:24:42.281357 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 05:24:42.281357 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 05:24:42.281357 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 05:24:42.281357 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 05:24:42.281357 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 05:24:42.281357 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 05:24:42.281357 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Jul 25 05:24:42.281357 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 410/803] 10.222.0.33 () {56 vars in 1096 bytes} [Mon Jul 25 05:24:42 2022] GET /identity/v3/roles/admin => generated 83 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:24:42.282057 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c002fbd-c6a3-4056-8831-d6a9a72d1723 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:24:42.299829 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bae1219-282a-4638-9844-9ba14478d2a2 None admin] RBAC: Authorizing `identity:create_grant(project_id=92afba266cce4d1bad4134c351e74be3, user_id=3352531be1b54f76b864383ccc2e5ec2, role_id=200e95cafbc54269a51756ddd406082f)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:24:42.300997 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bae1219-282a-4638-9844-9ba14478d2a2 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:24:42.315545 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 394/804] 10.222.0.33 () {58 vars in 1439 bytes} [Mon Jul 25 05:24:42 2022] PUT /identity/v3/projects/92afba266cce4d1bad4134c351e74be3/users/3352531be1b54f76b864383ccc2e5ec2/roles/200e95cafbc54269a51756ddd406082f => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 05:24:42.317129 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ea5dab-770e-46d9-a6bf-71070b7e2efb None None] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:24:42.317606 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ea5dab-770e-46d9-a6bf-71070b7e2efb None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:24:42.318100 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ea5dab-770e-46d9-a6bf-71070b7e2efb None None] PATH_INFO: `/` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:24:42.319623 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 395/805] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jul 25 05:24:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:24:42.320560 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ea5dab-770e-46d9-a6bf-71070b7e2efb None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:24:42.336208 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:24:42.368596 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:24:42.407963 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c002fbd-c6a3-4056-8831-d6a9a72d1723 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:24:42.460035 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e80f48-c16a-40b2-ab15-dcedb7f49eb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:24:42.461770 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e80f48-c16a-40b2-ab15-dcedb7f49eb0 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:24:42.462538 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e80f48-c16a-40b2-ab15-dcedb7f49eb0 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:24:42.463011 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e80f48-c16a-40b2-ab15-dcedb7f49eb0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:24:42.468681 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ea5dab-770e-46d9-a6bf-71070b7e2efb None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:24:42.479796 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46e80f48-c16a-40b2-ab15-dcedb7f49eb0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:24:42.482562 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46e80f48-c16a-40b2-ab15-dcedb7f49eb0 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:24:42.506522 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application [None req-46e80f48-c16a-40b2-ab15-dcedb7f49eb0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 05:24:42.506522 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:24:42.506522 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:24:42.506522 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:24:42.506522 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:24:42.506522 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:24:42.506522 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:24:42.506522 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:24:42.506522 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:24:42.506522 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:24:42.506522 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:24:42.506522 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:24:42.506522 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 05:24:42.506522 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 05:24:42.506522 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 05:24:42.507474 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 05:24:42.507474 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:24:42.507474 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:24:42.507474 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:24:42.507474 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:24:42.507474 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:24:42.507474 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:24:42.507474 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:24:42.507474 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application with Lock( Jul 25 05:24:42.507474 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:24:42.507474 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:24:42.507474 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:24:42.507474 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:24:42.507474 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:24:42.507474 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:24:42.507474 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:24:42.507474 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:24:42.508379 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 05:24:42.508379 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 05:24:42.508379 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 05:24:42.508379 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Jul 25 05:24:42.508610 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b0d1a8-0cde-4a67-a049-ed14dca4f59f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:24:42.508610 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b0d1a8-0cde-4a67-a049-ed14dca4f59f None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:24:42.508911 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b0d1a8-0cde-4a67-a049-ed14dca4f59f None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:24:42.509689 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b0d1a8-0cde-4a67-a049-ed14dca4f59f None admin] PATH_INFO: `/v3/roles` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:24:42.510511 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47b0d1a8-0cde-4a67-a049-ed14dca4f59f None admin] RBAC: Adding query filter params (name=admin) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:24:42.511086 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47b0d1a8-0cde-4a67-a049-ed14dca4f59f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:24:42.511807 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 411/806] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Jul 25 05:24:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:24:42.513323 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b755d9-35c6-4023-ae65-a60ae1e3f1ea None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:24:42.513773 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b755d9-35c6-4023-ae65-a60ae1e3f1ea None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:24:42.514058 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b755d9-35c6-4023-ae65-a60ae1e3f1ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:24:42.517307 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47b0d1a8-0cde-4a67-a049-ed14dca4f59f None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:24:42.526344 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 396/807] 10.222.0.33 () {56 vars in 1099 bytes} [Mon Jul 25 05:24:42 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:24:42.527824 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b0d1a8-0cde-4a67-a049-ed14dca4f59f None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:24:42.532641 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-b5b755d9-35c6-4023-ae65-a60ae1e3f1ea None None] MFA Rules not processed for user `1eae540242d94c10b0ad33960a855f08`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:24:42.534260 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-b5b755d9-35c6-4023-ae65-a60ae1e3f1ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:24:42.609837 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 412/808] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jul 25 05:24:42 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 25 05:24:42.611066 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b755d9-35c6-4023-ae65-a60ae1e3f1ea None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:24:42.613165 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:24:42.640978 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b0d1a8-0cde-4a67-a049-ed14dca4f59f None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:24:42.661178 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:24:42.690330 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-754609a9-cfbc-400b-b83b-0741e93eaa8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:24:42.691007 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754609a9-cfbc-400b-b83b-0741e93eaa8b None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:24:42.691297 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754609a9-cfbc-400b-b83b-0741e93eaa8b None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:24:42.691581 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754609a9-cfbc-400b-b83b-0741e93eaa8b None admin] PATH_INFO: `/v3/domains` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:24:42.692317 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-754609a9-cfbc-400b-b83b-0741e93eaa8b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:24:42.692630 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-754609a9-cfbc-400b-b83b-0741e93eaa8b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:24:42.694069 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-754609a9-cfbc-400b-b83b-0741e93eaa8b None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:24:42.710843 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 397/809] 10.222.0.33 () {56 vars in 1109 bytes} [Mon Jul 25 05:24:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:24:42.710843 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-754609a9-cfbc-400b-b83b-0741e93eaa8b None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:24:42.731044 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b755d9-35c6-4023-ae65-a60ae1e3f1ea None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:24:42.774844 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa93e58-3238-42f6-81a3-e1a35eeedf26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:24:42.775797 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa93e58-3238-42f6-81a3-e1a35eeedf26 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:24:42.776188 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa93e58-3238-42f6-81a3-e1a35eeedf26 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:24:42.777132 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa93e58-3238-42f6-81a3-e1a35eeedf26 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:24:42.787869 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fa93e58-3238-42f6-81a3-e1a35eeedf26 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:24:42.789488 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fa93e58-3238-42f6-81a3-e1a35eeedf26 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:24:42.816326 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application [None req-0fa93e58-3238-42f6-81a3-e1a35eeedf26 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 05:24:42.816326 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:24:42.816326 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:24:42.816326 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:24:42.816326 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:24:42.816326 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:24:42.816326 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:24:42.816326 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:24:42.816326 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:24:42.816326 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:24:42.816326 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:24:42.816326 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:24:42.816326 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 05:24:42.816326 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 05:24:42.816326 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 05:24:42.817520 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 05:24:42.817520 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:24:42.817520 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:24:42.817520 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:24:42.817520 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:24:42.817520 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:24:42.817520 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:24:42.817520 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:24:42.817520 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application with Lock( Jul 25 05:24:42.817520 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:24:42.817520 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:24:42.817520 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:24:42.817520 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:24:42.817520 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:24:42.817520 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:24:42.817520 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:24:42.817520 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:24:42.818710 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 05:24:42.818710 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 05:24:42.818710 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 05:24:42.818710 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Jul 25 05:24:42.825093 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 413/810] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Jul 25 05:24:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:24:42.826236 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa93e58-3238-42f6-81a3-e1a35eeedf26 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:24:42.855779 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-754609a9-cfbc-400b-b83b-0741e93eaa8b None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:24:42.902337 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:24:42.911379 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb676232-a3bb-47c3-9794-b6002416688b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:24:42.912374 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb676232-a3bb-47c3-9794-b6002416688b None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:24:42.913338 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb676232-a3bb-47c3-9794-b6002416688b None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:24:42.913791 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb676232-a3bb-47c3-9794-b6002416688b None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:24:42.914593 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb676232-a3bb-47c3-9794-b6002416688b None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:24:42.917022 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb676232-a3bb-47c3-9794-b6002416688b None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:24:42.935434 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application [None req-fb676232-a3bb-47c3-9794-b6002416688b None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 25 05:24:42.935434 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:24:42.935434 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 25 05:24:42.935434 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 25 05:24:42.935434 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 25 05:24:42.935434 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 25 05:24:42.935434 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 25 05:24:42.935434 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 25 05:24:42.935434 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 25 05:24:42.935434 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Jul 25 05:24:42.935434 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 25 05:24:42.935434 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Jul 25 05:24:42.935434 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:24:42.935434 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:24:42.935434 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:24:42.935434 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:24:42.936611 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:24:42.936611 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:24:42.936611 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:24:42.936611 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:24:42.936611 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:24:42.936611 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:24:42.936611 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:24:42.936611 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 25 05:24:42.936611 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 25 05:24:42.936611 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 25 05:24:42.936611 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 25 05:24:42.936611 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:24:42.936611 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:24:42.936611 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:24:42.936611 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:24:42.936611 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:24:42.937579 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:24:42.937579 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:24:42.937579 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application with Lock( Jul 25 05:24:42.937579 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:24:42.937579 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:24:42.937579 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:24:42.937579 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:24:42.937579 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:24:42.937579 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:24:42.937579 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:24:42.937579 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:24:42.937579 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 25 05:24:42.937579 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 25 05:24:42.937579 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 25 05:24:42.937579 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Jul 25 05:24:42.940689 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:24:42.942202 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 398/811] 10.222.0.33 () {56 vars in 1129 bytes} [Mon Jul 25 05:24:42 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:24:42.943118 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb676232-a3bb-47c3-9794-b6002416688b None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:24:42.950130 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa93e58-3238-42f6-81a3-e1a35eeedf26 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:24:42.996311 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d73339-a1c0-4a79-a610-fec5b5491d2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:24:42.997393 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d73339-a1c0-4a79-a610-fec5b5491d2d None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:24:42.998065 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d73339-a1c0-4a79-a610-fec5b5491d2d None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:24:42.998504 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d73339-a1c0-4a79-a610-fec5b5491d2d None admin] PATH_INFO: `/v3/projects/service` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:24:43.007945 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50d73339-a1c0-4a79-a610-fec5b5491d2d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:24:43.010252 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50d73339-a1c0-4a79-a610-fec5b5491d2d None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:24:43.028816 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application [None req-50d73339-a1c0-4a79-a610-fec5b5491d2d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 05:24:43.028816 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:24:43.028816 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:24:43.028816 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:24:43.028816 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:24:43.028816 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:24:43.028816 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:24:43.028816 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:24:43.028816 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:24:43.028816 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:24:43.028816 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:24:43.028816 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:24:43.028816 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 25 05:24:43.028816 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 25 05:24:43.028816 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 25 05:24:43.029911 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 25 05:24:43.029911 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:24:43.029911 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:24:43.029911 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:24:43.029911 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:24:43.029911 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:24:43.029911 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:24:43.029911 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:24:43.029911 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application with Lock( Jul 25 05:24:43.029911 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:24:43.029911 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:24:43.029911 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:24:43.029911 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:24:43.029911 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:24:43.029911 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:24:43.029911 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:24:43.029911 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:24:43.031029 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 25 05:24:43.031029 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 25 05:24:43.031029 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 25 05:24:43.031029 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 25 05:24:43.031029 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 25 05:24:43.031029 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 25 05:24:43.031029 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 05:24:43.031029 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Jul 25 05:24:43.038397 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 414/812] 10.222.0.33 () {56 vars in 1111 bytes} [Mon Jul 25 05:24:42 2022] GET /identity/v3/projects/service => generated 88 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:24:43.038397 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d73339-a1c0-4a79-a610-fec5b5491d2d None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:24:43.062605 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb676232-a3bb-47c3-9794-b6002416688b None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:24:43.101745 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e71dbe9-3dd0-4f6c-ac15-edb96719ab89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:24:43.102919 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e71dbe9-3dd0-4f6c-ac15-edb96719ab89 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:24:43.103368 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e71dbe9-3dd0-4f6c-ac15-edb96719ab89 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:24:43.103988 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e71dbe9-3dd0-4f6c-ac15-edb96719ab89 None admin] PATH_INFO: `/v3/domains` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:24:43.108836 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e71dbe9-3dd0-4f6c-ac15-edb96719ab89 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:24:43.109318 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e71dbe9-3dd0-4f6c-ac15-edb96719ab89 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:24:43.111361 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e71dbe9-3dd0-4f6c-ac15-edb96719ab89 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:24:43.129396 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 399/813] 10.222.0.33 () {56 vars in 1109 bytes} [Mon Jul 25 05:24:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:24:43.130552 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e71dbe9-3dd0-4f6c-ac15-edb96719ab89 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:24:43.164754 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d73339-a1c0-4a79-a610-fec5b5491d2d None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:24:43.202349 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a6cb03f-e935-4506-a47d-1798c56a43a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:24:43.203225 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6cb03f-e935-4506-a47d-1798c56a43a7 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:24:43.203539 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6cb03f-e935-4506-a47d-1798c56a43a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:24:43.203921 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6cb03f-e935-4506-a47d-1798c56a43a7 None admin] PATH_INFO: `/v3/services` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:24:43.204567 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:24:43.204852 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a6cb03f-e935-4506-a47d-1798c56a43a7 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:24:43.205804 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a6cb03f-e935-4506-a47d-1798c56a43a7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:24:43.207386 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a6cb03f-e935-4506-a47d-1798c56a43a7 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:24:43.217286 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 415/814] 10.222.0.33 () {56 vars in 1124 bytes} [Mon Jul 25 05:24:43 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:24:43.219144 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a6cb03f-e935-4506-a47d-1798c56a43a7 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:24:43.246723 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:24:43.263151 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e71dbe9-3dd0-4f6c-ac15-edb96719ab89 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:24:43.306810 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ec66de-14fa-4e23-9654-8ac97d17512b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:24:43.307741 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ec66de-14fa-4e23-9654-8ac97d17512b None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:24:43.308169 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ec66de-14fa-4e23-9654-8ac97d17512b None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:24:43.308588 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ec66de-14fa-4e23-9654-8ac97d17512b None admin] PATH_INFO: `/v3/projects` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:24:43.310007 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89ec66de-14fa-4e23-9654-8ac97d17512b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:24:43.310467 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89ec66de-14fa-4e23-9654-8ac97d17512b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:24:43.312384 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89ec66de-14fa-4e23-9654-8ac97d17512b None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:24:43.328151 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 400/815] 10.222.0.33 () {56 vars in 1148 bytes} [Mon Jul 25 05:24:43 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:24:43.329246 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ec66de-14fa-4e23-9654-8ac97d17512b None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:24:43.342010 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a6cb03f-e935-4506-a47d-1798c56a43a7 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:24:43.379821 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a94446-28ba-49da-86ee-21c951bd5514 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:24:43.380823 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a94446-28ba-49da-86ee-21c951bd5514 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:24:43.381225 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a94446-28ba-49da-86ee-21c951bd5514 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:24:43.381599 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a94446-28ba-49da-86ee-21c951bd5514 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:24:43.391960 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89a94446-28ba-49da-86ee-21c951bd5514 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:24:43.393675 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89a94446-28ba-49da-86ee-21c951bd5514 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:24:43.411814 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application [None req-89a94446-28ba-49da-86ee-21c951bd5514 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 25 05:24:43.411814 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:24:43.411814 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:24:43.411814 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:24:43.411814 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:24:43.411814 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:24:43.411814 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:24:43.411814 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:24:43.411814 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:24:43.411814 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:24:43.411814 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:24:43.411814 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:24:43.411814 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 25 05:24:43.411814 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 25 05:24:43.411814 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 25 05:24:43.411814 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 25 05:24:43.413017 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:24:43.413017 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:24:43.413017 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 25 05:24:43.413017 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 05:24:43.413017 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 25 05:24:43.413017 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 05:24:43.413017 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:24:43.413017 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:24:43.413017 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:24:43.413017 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:24:43.413017 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:24:43.413017 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application with Lock( Jul 25 05:24:43.413017 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:24:43.413017 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:24:43.413017 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:24:43.413017 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:24:43.413017 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:24:43.413017 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:24:43.414180 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:24:43.414180 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:24:43.414180 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 25 05:24:43.414180 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 25 05:24:43.414180 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 25 05:24:43.414180 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 25 05:24:43.414180 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 25 05:24:43.414180 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 25 05:24:43.414180 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 25 05:24:43.414180 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Jul 25 05:24:43.417368 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 416/816] 10.222.0.33 () {56 vars in 1093 bytes} [Mon Jul 25 05:24:43 2022] GET /identity/v3/users/nova => generated 82 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:24:43.418525 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a94446-28ba-49da-86ee-21c951bd5514 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:24:43.453708 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ec66de-14fa-4e23-9654-8ac97d17512b None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:24:43.473990 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:24:43.503926 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca93162-0f12-4949-b6d4-a23419d053ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:24:43.505043 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca93162-0f12-4949-b6d4-a23419d053ca None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:24:43.505560 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca93162-0f12-4949-b6d4-a23419d053ca None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:24:43.506043 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca93162-0f12-4949-b6d4-a23419d053ca None admin] PATH_INFO: `/v3/services` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:24:43.507027 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ca93162-0f12-4949-b6d4-a23419d053ca None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:24:43.508165 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ca93162-0f12-4949-b6d4-a23419d053ca None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:24:43.510309 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ca93162-0f12-4949-b6d4-a23419d053ca None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:24:43.515886 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:24:43.521999 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 401/817] 10.222.0.33 () {56 vars in 1124 bytes} [Mon Jul 25 05:24:43 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:24:43.523227 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca93162-0f12-4949-b6d4-a23419d053ca None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:24:43.539146 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a94446-28ba-49da-86ee-21c951bd5514 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:24:43.576256 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a3bafd6-7f5a-4293-b6b6-bd7cfe499387 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:24:43.577215 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3bafd6-7f5a-4293-b6b6-bd7cfe499387 None admin] REQUEST_METHOD: `PUT` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:24:43.577502 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3bafd6-7f5a-4293-b6b6-bd7cfe499387 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:24:43.577838 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3bafd6-7f5a-4293-b6b6-bd7cfe499387 None admin] PATH_INFO: `/v3/projects/92afba266cce4d1bad4134c351e74be3/users/eb135a8d5d7444cfbfa336115ba2ba44/roles/39e5c666285a42fe8fc2c3afca6260d1` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:24:43.579957 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a3bafd6-7f5a-4293-b6b6-bd7cfe499387 None admin] RBAC: Authorizing `identity:create_grant(project_id=92afba266cce4d1bad4134c351e74be3, user_id=eb135a8d5d7444cfbfa336115ba2ba44, role_id=39e5c666285a42fe8fc2c3afca6260d1)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:24:43.581434 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a3bafd6-7f5a-4293-b6b6-bd7cfe499387 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:24:43.602928 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 417/818] 10.222.0.33 () {58 vars in 1439 bytes} [Mon Jul 25 05:24:43 2022] PUT /identity/v3/projects/92afba266cce4d1bad4134c351e74be3/users/eb135a8d5d7444cfbfa336115ba2ba44/roles/39e5c666285a42fe8fc2c3afca6260d1 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 05:24:43.605489 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a3bafd6-7f5a-4293-b6b6-bd7cfe499387 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:24:43.694226 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca93162-0f12-4949-b6d4-a23419d053ca None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:24:43.763403 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a3bafd6-7f5a-4293-b6b6-bd7cfe499387 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:24:43.786638 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3902e5bd-94ab-41d8-8706-cf903f1f5f38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:24:43.787810 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3902e5bd-94ab-41d8-8706-cf903f1f5f38 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:24:43.788305 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3902e5bd-94ab-41d8-8706-cf903f1f5f38 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:24:43.788815 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3902e5bd-94ab-41d8-8706-cf903f1f5f38 None admin] PATH_INFO: `/v3/users` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:24:43.789893 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3902e5bd-94ab-41d8-8706-cf903f1f5f38 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:24:43.790416 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3902e5bd-94ab-41d8-8706-cf903f1f5f38 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:24:43.792201 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3902e5bd-94ab-41d8-8706-cf903f1f5f38 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:24:43.833789 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 402/819] 10.222.0.33 () {56 vars in 1133 bytes} [Mon Jul 25 05:24:43 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 311 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:24:43.838610 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0a8b5b-d8eb-48e5-b018-f23e9657cfd9 None None] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:24:43.840876 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0a8b5b-d8eb-48e5-b018-f23e9657cfd9 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:24:43.841382 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0a8b5b-d8eb-48e5-b018-f23e9657cfd9 None None] PATH_INFO: `/` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:24:43.844685 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 403/820] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jul 25 05:24:43 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:24:43.845687 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0a8b5b-d8eb-48e5-b018-f23e9657cfd9 None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:24:43.851762 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae08e5f-1af1-44e4-847b-100a31806b0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:24:43.852710 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae08e5f-1af1-44e4-847b-100a31806b0c None admin] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:24:43.854314 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae08e5f-1af1-44e4-847b-100a31806b0c None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:24:43.854785 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae08e5f-1af1-44e4-847b-100a31806b0c None admin] PATH_INFO: `/v3/endpoints` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:24:43.857525 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ae08e5f-1af1-44e4-847b-100a31806b0c None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:24:43.859167 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ae08e5f-1af1-44e4-847b-100a31806b0c None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:24:43.909379 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:24:43.913214 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:24:43.930755 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 418/821] 10.222.0.33 () {60 vars in 1144 bytes} [Mon Jul 25 05:24:43 2022] POST /identity/v3/endpoints => generated 346 bytes in 326 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 05:24:43.932133 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa24a256-cc6b-48cb-9a27-3cc5f7eb2755 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:24:43.932461 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa24a256-cc6b-48cb-9a27-3cc5f7eb2755 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:24:43.933221 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa24a256-cc6b-48cb-9a27-3cc5f7eb2755 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:24:43.981236 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-aa24a256-cc6b-48cb-9a27-3cc5f7eb2755 None None] MFA Rules not processed for user `1eae540242d94c10b0ad33960a855f08`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:24:43.982351 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-aa24a256-cc6b-48cb-9a27-3cc5f7eb2755 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:24:44.008135 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0a8b5b-d8eb-48e5-b018-f23e9657cfd9 None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:24:44.155202 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a095e29-f091-4c36-a1eb-5ee133ad2c05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:24:44.155875 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a095e29-f091-4c36-a1eb-5ee133ad2c05 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:24:44.156164 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a095e29-f091-4c36-a1eb-5ee133ad2c05 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:24:44.156446 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a095e29-f091-4c36-a1eb-5ee133ad2c05 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:24:44.179913 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 419/822] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jul 25 05:24:43 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 247 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 25 05:24:44.217648 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa24a256-cc6b-48cb-9a27-3cc5f7eb2755 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:24:44.296556 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a095e29-f091-4c36-a1eb-5ee133ad2c05 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:24:44.299671 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a095e29-f091-4c36-a1eb-5ee133ad2c05 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:24:44.369070 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa24a256-cc6b-48cb-9a27-3cc5f7eb2755 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:24:44.395367 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:24:44.399249 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:24:44.413739 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49381ce9-1949-4d75-a879-de9bfb2b5bfe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:24:44.414564 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49381ce9-1949-4d75-a879-de9bfb2b5bfe None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:24:44.414856 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49381ce9-1949-4d75-a879-de9bfb2b5bfe None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:24:44.415143 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49381ce9-1949-4d75-a879-de9bfb2b5bfe None admin] PATH_INFO: `/v3/services/network` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:24:44.415823 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49381ce9-1949-4d75-a879-de9bfb2b5bfe None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:24:44.419333 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49381ce9-1949-4d75-a879-de9bfb2b5bfe None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:24:44.439644 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application [None req-49381ce9-1949-4d75-a879-de9bfb2b5bfe None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jul 25 05:24:44.439644 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:24:44.439644 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 25 05:24:44.439644 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 25 05:24:44.439644 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 25 05:24:44.439644 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 25 05:24:44.439644 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 25 05:24:44.439644 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 25 05:24:44.439644 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 25 05:24:44.439644 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Jul 25 05:24:44.439644 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 25 05:24:44.439644 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Jul 25 05:24:44.439644 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:24:44.439644 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:24:44.439644 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:24:44.439644 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:24:44.439644 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:24:44.440729 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:24:44.440729 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:24:44.440729 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:24:44.440729 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:24:44.440729 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:24:44.440729 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:24:44.440729 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 25 05:24:44.440729 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 25 05:24:44.440729 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 25 05:24:44.440729 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 25 05:24:44.440729 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:24:44.440729 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:24:44.440729 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:24:44.440729 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:24:44.440729 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:24:44.440729 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:24:44.441624 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:24:44.441624 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application with Lock( Jul 25 05:24:44.441624 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:24:44.441624 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:24:44.441624 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:24:44.441624 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:24:44.441624 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:24:44.441624 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:24:44.441624 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:24:44.441624 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:24:44.441624 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 25 05:24:44.441624 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 25 05:24:44.441624 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 25 05:24:44.441624 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Jul 25 05:24:44.443627 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 420/823] 10.222.0.33 () {56 vars in 1111 bytes} [Mon Jul 25 05:24:44 2022] GET /identity/v3/services/network => generated 88 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 25 05:24:44.448926 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49381ce9-1949-4d75-a879-de9bfb2b5bfe None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:24:44.527507 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 404/824] 10.222.0.33 () {58 vars in 1282 bytes} [Mon Jul 25 05:24:43 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 682 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 25 05:24:44.533236 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a095e29-f091-4c36-a1eb-5ee133ad2c05 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:24:44.644544 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49381ce9-1949-4d75-a879-de9bfb2b5bfe None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:24:44.681637 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f1d66b-0fa2-47b7-97ba-6d41fbb19af9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:24:44.682732 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f1d66b-0fa2-47b7-97ba-6d41fbb19af9 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:24:44.683225 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f1d66b-0fa2-47b7-97ba-6d41fbb19af9 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:24:44.683719 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f1d66b-0fa2-47b7-97ba-6d41fbb19af9 None admin] PATH_INFO: `/v3/services` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:24:44.684631 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65f1d66b-0fa2-47b7-97ba-6d41fbb19af9 None admin] RBAC: Adding query filter params (name=network) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:24:44.685166 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65f1d66b-0fa2-47b7-97ba-6d41fbb19af9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:24:44.686746 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65f1d66b-0fa2-47b7-97ba-6d41fbb19af9 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:24:44.692508 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 421/825] 10.222.0.33 () {56 vars in 1112 bytes} [Mon Jul 25 05:24:44 2022] GET /identity/v3/services?name=network => generated 124 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:24:44.693990 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249b8ed5-abb3-4168-a69d-8e3975bd513c None None] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:24:44.694485 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249b8ed5-abb3-4168-a69d-8e3975bd513c None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:24:44.694983 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249b8ed5-abb3-4168-a69d-8e3975bd513c None None] PATH_INFO: `/` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:24:44.696645 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 422/826] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jul 25 05:24:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:24:44.697616 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-249b8ed5-abb3-4168-a69d-8e3975bd513c None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:24:44.717824 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a095e29-f091-4c36-a1eb-5ee133ad2c05 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:24:44.758564 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec4f8fd-40ee-4c92-8522-2a5f374f525a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:24:44.759701 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec4f8fd-40ee-4c92-8522-2a5f374f525a None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:24:44.760198 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec4f8fd-40ee-4c92-8522-2a5f374f525a None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:24:44.760675 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec4f8fd-40ee-4c92-8522-2a5f374f525a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:24:44.763428 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:24:44.768339 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:24:44.773196 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ec4f8fd-40ee-4c92-8522-2a5f374f525a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:24:44.774959 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ec4f8fd-40ee-4c92-8522-2a5f374f525a None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:24:44.795407 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application [None req-2ec4f8fd-40ee-4c92-8522-2a5f374f525a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 05:24:44.795407 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:24:44.795407 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:24:44.795407 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:24:44.795407 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:24:44.795407 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:24:44.795407 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:24:44.795407 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:24:44.795407 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:24:44.795407 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:24:44.795407 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:24:44.795407 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:24:44.795407 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 05:24:44.795407 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 05:24:44.795407 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 05:24:44.796380 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 05:24:44.796380 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:24:44.796380 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:24:44.796380 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:24:44.796380 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:24:44.796380 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:24:44.796380 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:24:44.796380 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:24:44.796380 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application with Lock( Jul 25 05:24:44.796380 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:24:44.796380 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:24:44.796380 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:24:44.796380 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:24:44.796380 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:24:44.796380 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:24:44.796380 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:24:44.796380 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:24:44.797303 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 05:24:44.797303 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 05:24:44.797303 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 05:24:44.797303 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Jul 25 05:24:44.799566 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 405/827] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Jul 25 05:24:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 267 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 25 05:24:44.801149 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283c2e04-def2-497f-90c9-e93f62d28e0d None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:24:44.801645 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283c2e04-def2-497f-90c9-e93f62d28e0d None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:24:44.802123 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283c2e04-def2-497f-90c9-e93f62d28e0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:24:44.820570 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-283c2e04-def2-497f-90c9-e93f62d28e0d None None] MFA Rules not processed for user `1eae540242d94c10b0ad33960a855f08`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:24:44.822230 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-283c2e04-def2-497f-90c9-e93f62d28e0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:24:44.827728 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-249b8ed5-abb3-4168-a69d-8e3975bd513c None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:24:44.872953 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf9d39e-221e-4416-95d0-12ec1dfb38d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:24:44.874046 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf9d39e-221e-4416-95d0-12ec1dfb38d8 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:24:44.874339 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf9d39e-221e-4416-95d0-12ec1dfb38d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:24:44.874619 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf9d39e-221e-4416-95d0-12ec1dfb38d8 None admin] PATH_INFO: `/v3/services` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:24:44.875495 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcf9d39e-221e-4416-95d0-12ec1dfb38d8 None admin] RBAC: Adding query filter params (type=network) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:24:44.875917 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcf9d39e-221e-4416-95d0-12ec1dfb38d8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:24:44.877325 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcf9d39e-221e-4416-95d0-12ec1dfb38d8 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:24:44.884349 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 423/828] 10.222.0.33 () {56 vars in 1112 bytes} [Mon Jul 25 05:24:44 2022] GET /identity/v3/services?type=network => generated 350 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:24:44.885171 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf9d39e-221e-4416-95d0-12ec1dfb38d8 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:24:44.902924 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 406/829] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jul 25 05:24:44 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 25 05:24:44.903702 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-283c2e04-def2-497f-90c9-e93f62d28e0d None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:24:45.011691 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf9d39e-221e-4416-95d0-12ec1dfb38d8 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:24:45.024576 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-283c2e04-def2-497f-90c9-e93f62d28e0d None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:24:45.042588 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:24:45.043096 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:24:45.047306 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e457f58d-70b2-419a-8ff5-2c34ed8089fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:24:45.048050 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e457f58d-70b2-419a-8ff5-2c34ed8089fd None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:24:45.048463 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e457f58d-70b2-419a-8ff5-2c34ed8089fd None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:24:45.048845 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e457f58d-70b2-419a-8ff5-2c34ed8089fd None admin] PATH_INFO: `/v3/domains` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:24:45.050881 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e457f58d-70b2-419a-8ff5-2c34ed8089fd None admin] RBAC: Adding query filter params (name=Default) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:24:45.051251 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e457f58d-70b2-419a-8ff5-2c34ed8089fd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:24:45.053015 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e457f58d-70b2-419a-8ff5-2c34ed8089fd None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:24:45.065291 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac68113-a519-4d4f-82b5-15958739963f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:24:45.066207 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac68113-a519-4d4f-82b5-15958739963f None admin] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:24:45.066729 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac68113-a519-4d4f-82b5-15958739963f None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:24:45.067191 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac68113-a519-4d4f-82b5-15958739963f None admin] PATH_INFO: `/v3/endpoints` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:24:45.068188 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dac68113-a519-4d4f-82b5-15958739963f None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:24:45.068590 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 424/830] 10.222.0.33 () {56 vars in 1109 bytes} [Mon Jul 25 05:24:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:24:45.069593 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e457f58d-70b2-419a-8ff5-2c34ed8089fd None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:24:45.070543 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dac68113-a519-4d4f-82b5-15958739963f None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:24:45.093514 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 407/831] 10.222.0.33 () {60 vars in 1144 bytes} [Mon Jul 25 05:24:44 2022] POST /identity/v3/endpoints => generated 327 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 05:24:45.094916 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac68113-a519-4d4f-82b5-15958739963f None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:24:45.185705 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e457f58d-70b2-419a-8ff5-2c34ed8089fd None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:24:45.218568 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac68113-a519-4d4f-82b5-15958739963f None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:24:45.256585 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4290a398-1cad-43e1-b225-21cf2595fa8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:24:45.257790 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4290a398-1cad-43e1-b225-21cf2595fa8b None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:24:45.258286 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4290a398-1cad-43e1-b225-21cf2595fa8b None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:24:45.258794 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4290a398-1cad-43e1-b225-21cf2595fa8b None admin] PATH_INFO: `/v3/projects/service` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:24:45.261780 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5335dfa4-df2e-49af-838e-fd28c36a0b7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:24:45.262780 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5335dfa4-df2e-49af-838e-fd28c36a0b7c None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:24:45.263011 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5335dfa4-df2e-49af-838e-fd28c36a0b7c None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:24:45.263298 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5335dfa4-df2e-49af-838e-fd28c36a0b7c None admin] PATH_INFO: `/v3/roles/service` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:24:45.268156 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4290a398-1cad-43e1-b225-21cf2595fa8b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:24:45.269961 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4290a398-1cad-43e1-b225-21cf2595fa8b None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:24:45.278573 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5335dfa4-df2e-49af-838e-fd28c36a0b7c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 05:24:45.285971 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application [None req-4290a398-1cad-43e1-b225-21cf2595fa8b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 05:24:45.285971 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:24:45.285971 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:24:45.285971 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:24:45.285971 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:24:45.285971 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:24:45.285971 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:24:45.285971 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:24:45.285971 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:24:45.285971 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:24:45.285971 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:24:45.285971 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:24:45.285971 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 25 05:24:45.285971 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 25 05:24:45.285971 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 25 05:24:45.286906 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 25 05:24:45.286906 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:24:45.286906 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:24:45.286906 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:24:45.286906 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:24:45.286906 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:24:45.286906 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:24:45.286906 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:24:45.286906 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application with Lock( Jul 25 05:24:45.286906 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:24:45.286906 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:24:45.286906 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:24:45.286906 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:24:45.286906 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:24:45.286906 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:24:45.286906 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:24:45.286906 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:24:45.287806 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 25 05:24:45.287806 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 25 05:24:45.287806 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 25 05:24:45.287806 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 25 05:24:45.287806 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 25 05:24:45.287806 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 25 05:24:45.287806 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 05:24:45.287806 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Jul 25 05:24:45.290552 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 408/832] 10.222.0.33 () {56 vars in 1111 bytes} [Mon Jul 25 05:24:45 2022] GET /identity/v3/projects/service => generated 88 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:24:45.297457 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4290a398-1cad-43e1-b225-21cf2595fa8b None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:24:45.299295 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application [None req-5335dfa4-df2e-49af-838e-fd28c36a0b7c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 25 05:24:45.299295 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:24:45.299295 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:24:45.299295 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:24:45.299295 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:24:45.299295 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:24:45.299295 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:24:45.299295 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:24:45.299295 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:24:45.299295 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:24:45.299295 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:24:45.299295 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:24:45.299295 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 05:24:45.299295 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 05:24:45.299295 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 05:24:45.299295 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 05:24:45.300217 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 05:24:45.300217 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 05:24:45.300217 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 05:24:45.300217 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 05:24:45.300217 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:24:45.300217 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:24:45.300217 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:24:45.300217 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:24:45.300217 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:24:45.300217 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:24:45.300217 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:24:45.300217 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application with Lock( Jul 25 05:24:45.300217 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:24:45.300217 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:24:45.300217 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:24:45.300217 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:24:45.301155 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:24:45.301155 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:24:45.301155 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:24:45.301155 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:24:45.301155 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 05:24:45.301155 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 05:24:45.301155 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 05:24:45.301155 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 05:24:45.301155 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 05:24:45.301155 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 05:24:45.301155 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 25 05:24:45.301155 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Jul 25 05:24:45.306364 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 425/833] 10.222.0.33 () {56 vars in 1102 bytes} [Mon Jul 25 05:24:45 2022] GET /identity/v3/roles/service => generated 85 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:24:45.312915 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5335dfa4-df2e-49af-838e-fd28c36a0b7c None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:24:45.363329 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:24:45.397213 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:24:45.447088 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5335dfa4-df2e-49af-838e-fd28c36a0b7c None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:24:45.491542 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f81d89-514f-40dc-b013-c3d4eb6dc4fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:24:45.492643 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f81d89-514f-40dc-b013-c3d4eb6dc4fa None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:24:45.493164 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f81d89-514f-40dc-b013-c3d4eb6dc4fa None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:24:45.493562 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f81d89-514f-40dc-b013-c3d4eb6dc4fa None admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:24:45.494413 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9f81d89-514f-40dc-b013-c3d4eb6dc4fa None admin] RBAC: Adding query filter params (name=service) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:24:45.494833 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9f81d89-514f-40dc-b013-c3d4eb6dc4fa None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:24:45.497289 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9f81d89-514f-40dc-b013-c3d4eb6dc4fa None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:24:45.508617 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 426/834] 10.222.0.33 () {56 vars in 1103 bytes} [Mon Jul 25 05:24:45 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:24:45.513072 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f81d89-514f-40dc-b013-c3d4eb6dc4fa None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:24:45.522701 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4290a398-1cad-43e1-b225-21cf2595fa8b None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:24:45.564037 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-706f6de6-31bf-4e2b-a991-4fd9eede4118 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:24:45.564816 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706f6de6-31bf-4e2b-a991-4fd9eede4118 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:24:45.565127 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706f6de6-31bf-4e2b-a991-4fd9eede4118 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:24:45.565411 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706f6de6-31bf-4e2b-a991-4fd9eede4118 None admin] PATH_INFO: `/v3/projects` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:24:45.566212 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-706f6de6-31bf-4e2b-a991-4fd9eede4118 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:24:45.566526 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-706f6de6-31bf-4e2b-a991-4fd9eede4118 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:24:45.568075 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-706f6de6-31bf-4e2b-a991-4fd9eede4118 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:24:45.581564 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 409/835] 10.222.0.33 () {56 vars in 1148 bytes} [Mon Jul 25 05:24:45 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 285 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 05:24:45.587172 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-706f6de6-31bf-4e2b-a991-4fd9eede4118 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:24:45.635631 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f81d89-514f-40dc-b013-c3d4eb6dc4fa None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:24:45.637412 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:24:45.674719 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5545e0ba-45d0-4d02-8864-414f4bb9e0f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:24:45.675559 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5545e0ba-45d0-4d02-8864-414f4bb9e0f5 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:24:45.675847 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5545e0ba-45d0-4d02-8864-414f4bb9e0f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:24:45.676133 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5545e0ba-45d0-4d02-8864-414f4bb9e0f5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:24:45.686857 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5545e0ba-45d0-4d02-8864-414f4bb9e0f5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:24:45.688322 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5545e0ba-45d0-4d02-8864-414f4bb9e0f5 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:24:45.692323 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:24:45.706192 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-706f6de6-31bf-4e2b-a991-4fd9eede4118 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:24:45.707433 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application [None req-5545e0ba-45d0-4d02-8864-414f4bb9e0f5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 05:24:45.707433 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:24:45.707433 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:24:45.707433 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:24:45.707433 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:24:45.707433 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:24:45.707433 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:24:45.707433 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:24:45.707433 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:24:45.707433 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:24:45.707433 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:24:45.707433 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:24:45.707433 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 05:24:45.707433 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 05:24:45.707433 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 05:24:45.708311 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 05:24:45.708311 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:24:45.708311 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:24:45.708311 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:24:45.708311 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:24:45.708311 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:24:45.708311 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:24:45.708311 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:24:45.708311 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application with Lock( Jul 25 05:24:45.708311 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:24:45.708311 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:24:45.708311 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:24:45.708311 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:24:45.708311 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:24:45.708311 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:24:45.708311 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:24:45.708311 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:24:45.709233 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 05:24:45.709233 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 05:24:45.709233 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 05:24:45.709233 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Jul 25 05:24:45.713377 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 427/836] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Jul 25 05:24:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:24:45.715097 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5545e0ba-45d0-4d02-8864-414f4bb9e0f5 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:24:45.746447 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0796e04d-e73e-40d9-8bcf-e07a81aa0cf8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:24:45.747483 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0796e04d-e73e-40d9-8bcf-e07a81aa0cf8 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:24:45.747964 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0796e04d-e73e-40d9-8bcf-e07a81aa0cf8 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:24:45.748437 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0796e04d-e73e-40d9-8bcf-e07a81aa0cf8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:24:45.749538 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0796e04d-e73e-40d9-8bcf-e07a81aa0cf8 None admin] RBAC: Adding query filter params (role.id=39e5c666285a42fe8fc2c3afca6260d1, scope.project.id=92afba266cce4d1bad4134c351e74be3, user.id=1cd482a61e8a420db45bb532e5dae6e2) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:24:45.750120 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0796e04d-e73e-40d9-8bcf-e07a81aa0cf8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:24:45.751789 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0796e04d-e73e-40d9-8bcf-e07a81aa0cf8 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:24:45.758533 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 410/837] 10.222.0.33 () {56 vars in 1374 bytes} [Mon Jul 25 05:24:45 2022] GET /identity/v3/role_assignments?user.id=1cd482a61e8a420db45bb532e5dae6e2&scope.project.id=92afba266cce4d1bad4134c351e74be3&role.id=39e5c666285a42fe8fc2c3afca6260d1 => generated 608 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:24:45.835913 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5545e0ba-45d0-4d02-8864-414f4bb9e0f5 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:24:45.875092 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52296cb1-4440-4848-8464-72bee1d5cc0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:24:45.875787 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52296cb1-4440-4848-8464-72bee1d5cc0d None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:24:45.876077 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52296cb1-4440-4848-8464-72bee1d5cc0d None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:24:45.876359 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52296cb1-4440-4848-8464-72bee1d5cc0d None admin] PATH_INFO: `/v3/domains` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:24:45.877143 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52296cb1-4440-4848-8464-72bee1d5cc0d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:24:45.877454 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52296cb1-4440-4848-8464-72bee1d5cc0d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:24:45.879163 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52296cb1-4440-4848-8464-72bee1d5cc0d None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:24:45.898220 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 428/838] 10.222.0.33 () {56 vars in 1109 bytes} [Mon Jul 25 05:24:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 05:24:45.899618 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0796e04d-e73e-40d9-8bcf-e07a81aa0cf8 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:24:45.960152 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552ebb7d-d6ef-4613-9eba-b17e396522ed None None] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:24:45.960398 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552ebb7d-d6ef-4613-9eba-b17e396522ed None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:24:45.960649 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552ebb7d-d6ef-4613-9eba-b17e396522ed None None] PATH_INFO: `/` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:24:45.962192 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 429/839] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jul 25 05:24:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 25 05:24:45.967558 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41762e36-f9e3-4ac4-86fb-55acb75a9258 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:24:45.967844 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41762e36-f9e3-4ac4-86fb-55acb75a9258 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:24:45.968133 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41762e36-f9e3-4ac4-86fb-55acb75a9258 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:24:45.986182 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-41762e36-f9e3-4ac4-86fb-55acb75a9258 None None] MFA Rules not processed for user `1eae540242d94c10b0ad33960a855f08`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:24:45.987408 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-41762e36-f9e3-4ac4-86fb-55acb75a9258 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:24:45.997692 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:24:46.017836 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0796e04d-e73e-40d9-8bcf-e07a81aa0cf8 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:24:46.064552 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbdc7611-fa10-4bd5-b90d-570a12458ff9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:24:46.066221 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbdc7611-fa10-4bd5-b90d-570a12458ff9 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:24:46.066723 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbdc7611-fa10-4bd5-b90d-570a12458ff9 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:24:46.067202 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbdc7611-fa10-4bd5-b90d-570a12458ff9 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:24:46.068889 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 430/840] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jul 25 05:24:45 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 25 05:24:46.077087 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbdc7611-fa10-4bd5-b90d-570a12458ff9 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:24:46.078869 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbdc7611-fa10-4bd5-b90d-570a12458ff9 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:24:46.086796 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41762e36-f9e3-4ac4-86fb-55acb75a9258 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:24:46.096397 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application [None req-fbdc7611-fa10-4bd5-b90d-570a12458ff9 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 25 05:24:46.096397 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:24:46.096397 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:24:46.096397 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:24:46.096397 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:24:46.096397 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:24:46.096397 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:24:46.096397 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:24:46.096397 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:24:46.096397 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:24:46.096397 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:24:46.096397 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:24:46.096397 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 25 05:24:46.096397 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 25 05:24:46.096397 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 25 05:24:46.096397 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 25 05:24:46.097392 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:24:46.097392 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:24:46.097392 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 25 05:24:46.097392 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 05:24:46.097392 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 25 05:24:46.097392 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 05:24:46.097392 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:24:46.097392 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:24:46.097392 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:24:46.097392 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:24:46.097392 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:24:46.097392 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application with Lock( Jul 25 05:24:46.097392 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:24:46.097392 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:24:46.097392 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:24:46.097392 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:24:46.097392 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:24:46.097392 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:24:46.098341 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:24:46.098341 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:24:46.098341 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 25 05:24:46.098341 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 25 05:24:46.098341 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 25 05:24:46.098341 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 25 05:24:46.098341 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 25 05:24:46.098341 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 25 05:24:46.098341 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 25 05:24:46.098341 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Jul 25 05:24:46.101299 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 411/841] 10.222.0.33 () {56 vars in 1117 bytes} [Mon Jul 25 05:24:45 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:24:46.107735 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbdc7611-fa10-4bd5-b90d-570a12458ff9 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:24:46.123320 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:24:46.216815 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41762e36-f9e3-4ac4-86fb-55acb75a9258 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:24:46.223089 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbdc7611-fa10-4bd5-b90d-570a12458ff9 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:24:46.256733 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad0914ed-c2c5-4dcb-9dfd-513efad8191c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:24:46.257860 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0914ed-c2c5-4dcb-9dfd-513efad8191c None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:24:46.258357 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0914ed-c2c5-4dcb-9dfd-513efad8191c None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:24:46.258633 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0914ed-c2c5-4dcb-9dfd-513efad8191c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:24:46.268116 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae88cd3-901d-4503-9bf4-cb9dec00c72b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:24:46.269038 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae88cd3-901d-4503-9bf4-cb9dec00c72b None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:24:46.269738 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae88cd3-901d-4503-9bf4-cb9dec00c72b None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:24:46.270111 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad0914ed-c2c5-4dcb-9dfd-513efad8191c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 05:24:46.270328 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae88cd3-901d-4503-9bf4-cb9dec00c72b None admin] PATH_INFO: `/v3/users` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:24:46.271323 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ae88cd3-901d-4503-9bf4-cb9dec00c72b None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:24:46.271790 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ae88cd3-901d-4503-9bf4-cb9dec00c72b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:24:46.273413 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ae88cd3-901d-4503-9bf4-cb9dec00c72b None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:24:46.278818 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application [None req-ad0914ed-c2c5-4dcb-9dfd-513efad8191c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 05:24:46.278818 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:24:46.278818 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:24:46.278818 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:24:46.278818 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:24:46.278818 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:24:46.278818 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:24:46.278818 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:24:46.278818 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:24:46.278818 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:24:46.278818 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:24:46.278818 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:24:46.278818 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 05:24:46.278818 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 05:24:46.278818 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 05:24:46.278818 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 05:24:46.279980 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 05:24:46.279980 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 05:24:46.279980 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 05:24:46.279980 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 05:24:46.279980 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:24:46.279980 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:24:46.279980 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:24:46.279980 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:24:46.279980 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:24:46.279980 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:24:46.279980 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:24:46.279980 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application with Lock( Jul 25 05:24:46.279980 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:24:46.279980 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:24:46.279980 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:24:46.279980 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:24:46.281078 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:24:46.281078 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:24:46.281078 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:24:46.281078 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:24:46.281078 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 05:24:46.281078 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 05:24:46.281078 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 05:24:46.281078 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 05:24:46.281078 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 05:24:46.281078 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 05:24:46.281078 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 05:24:46.281078 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Jul 25 05:24:46.282194 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 431/842] 10.222.0.33 () {56 vars in 1096 bytes} [Mon Jul 25 05:24:46 2022] GET /identity/v3/roles/admin => generated 83 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 25 05:24:46.287816 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad0914ed-c2c5-4dcb-9dfd-513efad8191c None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:24:46.294040 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:24:46.305502 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 412/843] 10.222.0.33 () {56 vars in 1149 bytes} [Mon Jul 25 05:24:46 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 05:24:46.305782 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c2d5a1-f6fa-439d-a2fe-3e9b6438f2ec None None] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:24:46.306072 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c2d5a1-f6fa-439d-a2fe-3e9b6438f2ec None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:24:46.306513 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c2d5a1-f6fa-439d-a2fe-3e9b6438f2ec None None] PATH_INFO: `/` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:24:46.308332 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 413/844] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jul 25 05:24:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:24:46.309000 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c2d5a1-f6fa-439d-a2fe-3e9b6438f2ec None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:24:46.384156 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:24:46.403095 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad0914ed-c2c5-4dcb-9dfd-513efad8191c None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:24:46.419421 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c2d5a1-f6fa-439d-a2fe-3e9b6438f2ec None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:24:46.440966 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d86ebed4-bf65-44fb-8fcb-f8bb9c97f73f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:24:46.441667 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86ebed4-bf65-44fb-8fcb-f8bb9c97f73f None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:24:46.441958 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86ebed4-bf65-44fb-8fcb-f8bb9c97f73f None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:24:46.442242 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86ebed4-bf65-44fb-8fcb-f8bb9c97f73f None admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:24:46.443018 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d86ebed4-bf65-44fb-8fcb-f8bb9c97f73f None admin] RBAC: Adding query filter params (name=admin) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:24:46.443333 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d86ebed4-bf65-44fb-8fcb-f8bb9c97f73f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:24:46.444743 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d86ebed4-bf65-44fb-8fcb-f8bb9c97f73f None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:24:46.459293 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 432/845] 10.222.0.33 () {56 vars in 1099 bytes} [Mon Jul 25 05:24:46 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:24:46.459293 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70b7eec-a9f3-4715-baf1-df8828c3a020 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:24:46.459293 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70b7eec-a9f3-4715-baf1-df8828c3a020 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:24:46.459293 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70b7eec-a9f3-4715-baf1-df8828c3a020 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:24:46.476740 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-f70b7eec-a9f3-4715-baf1-df8828c3a020 None None] MFA Rules not processed for user `1eae540242d94c10b0ad33960a855f08`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:24:46.477943 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-f70b7eec-a9f3-4715-baf1-df8828c3a020 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:24:46.545745 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:24:46.560271 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 433/846] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jul 25 05:24:46 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 25 05:24:46.560271 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f70b7eec-a9f3-4715-baf1-df8828c3a020 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:24:46.579588 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed6b897-72b6-45bb-ac70-03fd3e2b18d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:24:46.580251 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed6b897-72b6-45bb-ac70-03fd3e2b18d3 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:24:46.580557 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed6b897-72b6-45bb-ac70-03fd3e2b18d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:24:46.580842 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed6b897-72b6-45bb-ac70-03fd3e2b18d3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:24:46.654604 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ed6b897-72b6-45bb-ac70-03fd3e2b18d3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:24:46.656056 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ed6b897-72b6-45bb-ac70-03fd3e2b18d3 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:24:46.670868 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f70b7eec-a9f3-4715-baf1-df8828c3a020 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:24:46.706437 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a7b7023-8bda-498a-9ce1-919b635d42e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:24:46.707223 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7b7023-8bda-498a-9ce1-919b635d42e7 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:24:46.707515 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7b7023-8bda-498a-9ce1-919b635d42e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:24:46.707791 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7b7023-8bda-498a-9ce1-919b635d42e7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:24:46.718934 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a7b7023-8bda-498a-9ce1-919b635d42e7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:24:46.720365 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a7b7023-8bda-498a-9ce1-919b635d42e7 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:24:46.729811 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:24:46.741493 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application [None req-5a7b7023-8bda-498a-9ce1-919b635d42e7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 05:24:46.741493 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:24:46.741493 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:24:46.741493 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:24:46.741493 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:24:46.741493 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:24:46.741493 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:24:46.741493 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:24:46.741493 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:24:46.741493 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:24:46.741493 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:24:46.741493 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:24:46.741493 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 05:24:46.741493 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 05:24:46.741493 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 05:24:46.742409 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 05:24:46.742409 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:24:46.742409 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:24:46.742409 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:24:46.742409 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:24:46.742409 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:24:46.742409 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:24:46.742409 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:24:46.742409 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application with Lock( Jul 25 05:24:46.742409 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:24:46.742409 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:24:46.742409 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:24:46.742409 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:24:46.742409 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:24:46.742409 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:24:46.742409 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:24:46.742409 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:24:46.743303 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 05:24:46.743303 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 05:24:46.743303 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 05:24:46.743303 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Jul 25 05:24:46.743303 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 434/847] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Jul 25 05:24:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:24:46.743605 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a7b7023-8bda-498a-9ce1-919b635d42e7 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:24:46.764819 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 414/848] 10.222.0.33 () {58 vars in 1282 bytes} [Mon Jul 25 05:24:46 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 456 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 25 05:24:46.765702 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed6b897-72b6-45bb-ac70-03fd3e2b18d3 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:24:46.805088 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:24:46.859882 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a7b7023-8bda-498a-9ce1-919b635d42e7 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:24:46.878530 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed6b897-72b6-45bb-ac70-03fd3e2b18d3 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:24:46.896884 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-743303c4-c43c-4767-b8eb-7ea810815b62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:24:46.897682 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743303c4-c43c-4767-b8eb-7ea810815b62 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:24:46.897966 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743303c4-c43c-4767-b8eb-7ea810815b62 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:24:46.898242 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743303c4-c43c-4767-b8eb-7ea810815b62 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:24:46.898895 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-743303c4-c43c-4767-b8eb-7ea810815b62 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:24:46.901079 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-743303c4-c43c-4767-b8eb-7ea810815b62 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:24:46.914221 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb51aec-92a2-4693-bbc3-952cce39d5e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:24:46.914913 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb51aec-92a2-4693-bbc3-952cce39d5e8 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:24:46.915203 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb51aec-92a2-4693-bbc3-952cce39d5e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:24:46.915485 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb51aec-92a2-4693-bbc3-952cce39d5e8 None admin] PATH_INFO: `/v3/domains` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:24:46.915872 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application [None req-743303c4-c43c-4767-b8eb-7ea810815b62 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 25 05:24:46.915872 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:24:46.915872 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 25 05:24:46.915872 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 25 05:24:46.915872 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 25 05:24:46.915872 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 25 05:24:46.915872 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 25 05:24:46.915872 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 25 05:24:46.915872 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 25 05:24:46.915872 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Jul 25 05:24:46.915872 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 25 05:24:46.915872 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Jul 25 05:24:46.915872 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:24:46.915872 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:24:46.915872 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:24:46.915872 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:24:46.915872 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:24:46.916853 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:24:46.916853 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:24:46.916853 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:24:46.916853 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:24:46.916853 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:24:46.916853 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:24:46.916853 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 25 05:24:46.916853 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 25 05:24:46.916853 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 25 05:24:46.916853 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 25 05:24:46.916853 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:24:46.916853 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:24:46.916853 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:24:46.916853 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:24:46.916853 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:24:46.916853 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:24:46.923668 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:24:46.923668 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application with Lock( Jul 25 05:24:46.923668 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:24:46.923668 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:24:46.923668 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:24:46.923668 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:24:46.923668 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:24:46.923668 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:24:46.923668 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:24:46.923668 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:24:46.923668 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 25 05:24:46.923668 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 25 05:24:46.923668 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 25 05:24:46.923668 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Jul 25 05:24:46.924416 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bb51aec-92a2-4693-bbc3-952cce39d5e8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:24:46.924416 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bb51aec-92a2-4693-bbc3-952cce39d5e8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:24:46.924546 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 435/849] 10.222.0.33 () {56 vars in 1114 bytes} [Mon Jul 25 05:24:46 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:24:46.924624 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bb51aec-92a2-4693-bbc3-952cce39d5e8 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:24:46.924697 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-743303c4-c43c-4767-b8eb-7ea810815b62 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:24:46.930294 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 415/850] 10.222.0.33 () {56 vars in 1109 bytes} [Mon Jul 25 05:24:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:24:46.936215 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb51aec-92a2-4693-bbc3-952cce39d5e8 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:24:46.968888 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:24:47.039749 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-743303c4-c43c-4767-b8eb-7ea810815b62 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:24:47.046618 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb51aec-92a2-4693-bbc3-952cce39d5e8 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:24:47.053343 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:24:47.076196 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc21fa4-a15b-4df1-93a8-f7f76d013996 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:24:47.077027 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc21fa4-a15b-4df1-93a8-f7f76d013996 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:24:47.077305 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc21fa4-a15b-4df1-93a8-f7f76d013996 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:24:47.077587 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc21fa4-a15b-4df1-93a8-f7f76d013996 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:24:47.088232 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bc21fa4-a15b-4df1-93a8-f7f76d013996 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:24:47.088359 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb77765f-5639-464f-91da-843bf370214e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:24:47.089602 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb77765f-5639-464f-91da-843bf370214e None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:24:47.089903 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb77765f-5639-464f-91da-843bf370214e None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:24:47.090182 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb77765f-5639-464f-91da-843bf370214e None admin] PATH_INFO: `/v3/services` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:24:47.090346 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bc21fa4-a15b-4df1-93a8-f7f76d013996 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:24:47.090892 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb77765f-5639-464f-91da-843bf370214e None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:24:47.091188 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb77765f-5639-464f-91da-843bf370214e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:24:47.092537 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb77765f-5639-464f-91da-843bf370214e None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:24:47.097765 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 416/851] 10.222.0.33 () {56 vars in 1114 bytes} [Mon Jul 25 05:24:46 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:24:47.102842 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb77765f-5639-464f-91da-843bf370214e None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:24:47.109182 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application [None req-9bc21fa4-a15b-4df1-93a8-f7f76d013996 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 05:24:47.109182 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:24:47.109182 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:24:47.109182 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:24:47.109182 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:24:47.109182 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:24:47.109182 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:24:47.109182 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:24:47.109182 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:24:47.109182 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:24:47.109182 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:24:47.109182 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:24:47.109182 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 05:24:47.109182 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 05:24:47.109182 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 05:24:47.110099 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 05:24:47.110099 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:24:47.110099 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:24:47.110099 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:24:47.110099 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:24:47.110099 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:24:47.110099 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:24:47.110099 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:24:47.110099 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application with Lock( Jul 25 05:24:47.110099 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:24:47.110099 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:24:47.110099 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:24:47.110099 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:24:47.110099 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:24:47.110099 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:24:47.110099 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:24:47.110099 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:24:47.112052 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 05:24:47.112052 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 05:24:47.112052 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 05:24:47.112052 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Jul 25 05:24:47.112052 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 436/852] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Jul 25 05:24:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:24:47.112052 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc21fa4-a15b-4df1-93a8-f7f76d013996 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:24:47.215048 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:24:47.223473 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb77765f-5639-464f-91da-843bf370214e None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:24:47.245208 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc21fa4-a15b-4df1-93a8-f7f76d013996 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:24:47.262975 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-983289d7-dea8-4c0b-b030-0f7cdb5a081e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:24:47.263775 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983289d7-dea8-4c0b-b030-0f7cdb5a081e None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:24:47.264062 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983289d7-dea8-4c0b-b030-0f7cdb5a081e None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:24:47.264338 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983289d7-dea8-4c0b-b030-0f7cdb5a081e None admin] PATH_INFO: `/v3/users/swift` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:24:47.271945 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-983289d7-dea8-4c0b-b030-0f7cdb5a081e None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:24:47.273439 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-983289d7-dea8-4c0b-b030-0f7cdb5a081e None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:24:47.288512 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d5034c9-a223-4321-ac73-7b0b766439c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:24:47.289161 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application [None req-983289d7-dea8-4c0b-b030-0f7cdb5a081e None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 25 05:24:47.289161 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:24:47.289161 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:24:47.289161 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:24:47.289161 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:24:47.289161 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:24:47.289161 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:24:47.289161 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:24:47.289161 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:24:47.289161 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:24:47.289161 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:24:47.289161 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:24:47.289161 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 25 05:24:47.289161 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 25 05:24:47.289161 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 25 05:24:47.289161 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 25 05:24:47.290225 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:24:47.290225 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:24:47.290225 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 25 05:24:47.290225 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 05:24:47.290225 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 25 05:24:47.290225 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 05:24:47.290225 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:24:47.290225 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:24:47.290225 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:24:47.290225 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:24:47.290225 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:24:47.290225 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application with Lock( Jul 25 05:24:47.290225 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:24:47.290225 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:24:47.290225 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:24:47.290225 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:24:47.290225 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:24:47.290225 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:24:47.297048 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:24:47.297048 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:24:47.297048 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 25 05:24:47.297048 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 25 05:24:47.297048 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 25 05:24:47.297048 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 25 05:24:47.297048 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 25 05:24:47.297048 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 25 05:24:47.297048 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 25 05:24:47.297048 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Jul 25 05:24:47.297639 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5034c9-a223-4321-ac73-7b0b766439c8 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:24:47.297639 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5034c9-a223-4321-ac73-7b0b766439c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:24:47.297639 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5034c9-a223-4321-ac73-7b0b766439c8 None admin] PATH_INFO: `/v3/services` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:24:47.297639 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d5034c9-a223-4321-ac73-7b0b766439c8 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:24:47.297889 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 417/853] 10.222.0.33 () {56 vars in 1096 bytes} [Mon Jul 25 05:24:47 2022] GET /identity/v3/users/swift => generated 83 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:24:47.297959 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d5034c9-a223-4321-ac73-7b0b766439c8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:24:47.298028 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-983289d7-dea8-4c0b-b030-0f7cdb5a081e None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:24:47.298107 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d5034c9-a223-4321-ac73-7b0b766439c8 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:24:47.313023 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 437/854] 10.222.0.33 () {56 vars in 1114 bytes} [Mon Jul 25 05:24:47 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:24:47.313023 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d5034c9-a223-4321-ac73-7b0b766439c8 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:24:47.355843 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:24:47.453661 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-983289d7-dea8-4c0b-b030-0f7cdb5a081e None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:24:47.468076 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d5034c9-a223-4321-ac73-7b0b766439c8 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:24:47.517341 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-047efc06-c75a-41f2-a395-d105c5b8ec32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:24:47.518035 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047efc06-c75a-41f2-a395-d105c5b8ec32 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:24:47.518320 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047efc06-c75a-41f2-a395-d105c5b8ec32 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:24:47.518595 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047efc06-c75a-41f2-a395-d105c5b8ec32 None admin] PATH_INFO: `/v3/users` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:24:47.519451 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-047efc06-c75a-41f2-a395-d105c5b8ec32 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:24:47.519762 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-047efc06-c75a-41f2-a395-d105c5b8ec32 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:24:47.521279 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-047efc06-c75a-41f2-a395-d105c5b8ec32 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:24:47.526047 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ca19c4-7ac2-4795-91ea-ef28d613aa34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:24:47.527408 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ca19c4-7ac2-4795-91ea-ef28d613aa34 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:24:47.527698 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ca19c4-7ac2-4795-91ea-ef28d613aa34 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:24:47.527978 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ca19c4-7ac2-4795-91ea-ef28d613aa34 None admin] PATH_INFO: `/v3/domains` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:24:47.528732 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5ca19c4-7ac2-4795-91ea-ef28d613aa34 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:24:47.529082 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5ca19c4-7ac2-4795-91ea-ef28d613aa34 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:24:47.530487 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5ca19c4-7ac2-4795-91ea-ef28d613aa34 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:24:47.551776 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 418/855] 10.222.0.33 () {56 vars in 1109 bytes} [Mon Jul 25 05:24:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:24:47.553851 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 438/856] 10.222.0.33 () {56 vars in 1135 bytes} [Mon Jul 25 05:24:47 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:24:47.562795 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ca19c4-7ac2-4795-91ea-ef28d613aa34 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:24:47.567995 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-047efc06-c75a-41f2-a395-d105c5b8ec32 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:24:47.580702 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:24:47.691169 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:24:47.714610 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ca19c4-7ac2-4795-91ea-ef28d613aa34 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:24:47.716230 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-047efc06-c75a-41f2-a395-d105c5b8ec32 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:24:47.758196 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92008949-1f3a-4f52-9235-3b517e8d1da0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:24:47.759238 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92008949-1f3a-4f52-9235-3b517e8d1da0 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:24:47.759680 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92008949-1f3a-4f52-9235-3b517e8d1da0 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:24:47.760054 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92008949-1f3a-4f52-9235-3b517e8d1da0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:24:47.763224 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db596e2-62a4-442b-be65-cbe0db10cc69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:24:47.763892 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db596e2-62a4-442b-be65-cbe0db10cc69 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:24:47.764625 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db596e2-62a4-442b-be65-cbe0db10cc69 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:24:47.764954 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db596e2-62a4-442b-be65-cbe0db10cc69 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:24:47.771701 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92008949-1f3a-4f52-9235-3b517e8d1da0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:24:47.773428 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92008949-1f3a-4f52-9235-3b517e8d1da0 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:24:47.791177 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application [None req-92008949-1f3a-4f52-9235-3b517e8d1da0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 05:24:47.791177 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:24:47.791177 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:24:47.791177 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:24:47.791177 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:24:47.791177 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:24:47.791177 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:24:47.791177 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:24:47.791177 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:24:47.791177 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:24:47.791177 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:24:47.791177 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:24:47.791177 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 25 05:24:47.791177 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 25 05:24:47.791177 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 25 05:24:47.793087 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 25 05:24:47.793087 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:24:47.793087 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:24:47.793087 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:24:47.793087 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:24:47.793087 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:24:47.793087 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:24:47.793087 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:24:47.793087 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application with Lock( Jul 25 05:24:47.793087 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:24:47.793087 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:24:47.793087 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:24:47.793087 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:24:47.793087 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:24:47.793087 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:24:47.793087 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:24:47.793087 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:24:47.794848 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 25 05:24:47.794848 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 25 05:24:47.794848 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 25 05:24:47.794848 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 25 05:24:47.794848 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 25 05:24:47.794848 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 25 05:24:47.794848 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 05:24:47.794848 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Jul 25 05:24:47.794848 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 419/857] 10.222.0.33 () {56 vars in 1111 bytes} [Mon Jul 25 05:24:47 2022] GET /identity/v3/projects/service => generated 88 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:24:47.801203 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92008949-1f3a-4f52-9235-3b517e8d1da0 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:24:47.861161 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6db596e2-62a4-442b-be65-cbe0db10cc69 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:24:47.862636 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6db596e2-62a4-442b-be65-cbe0db10cc69 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:24:47.923485 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:24:47.959988 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92008949-1f3a-4f52-9235-3b517e8d1da0 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:24:47.981332 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:24:47.996575 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e5e4333-a370-4c77-bb03-80f595e02a74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:24:47.997176 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5e4333-a370-4c77-bb03-80f595e02a74 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:24:47.997535 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5e4333-a370-4c77-bb03-80f595e02a74 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:24:47.997804 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5e4333-a370-4c77-bb03-80f595e02a74 None admin] PATH_INFO: `/v3/projects` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:24:47.998502 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e5e4333-a370-4c77-bb03-80f595e02a74 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:24:47.998809 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e5e4333-a370-4c77-bb03-80f595e02a74 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:24:48.000430 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e5e4333-a370-4c77-bb03-80f595e02a74 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:24:48.000538 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 439/858] 10.222.0.33 () {58 vars in 1282 bytes} [Mon Jul 25 05:24:47 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 433 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 25 05:24:48.006051 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbab33e4-8491-4d94-9c41-bf3b518ee373 None None] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:24:48.006345 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbab33e4-8491-4d94-9c41-bf3b518ee373 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:24:48.006625 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbab33e4-8491-4d94-9c41-bf3b518ee373 None None] PATH_INFO: `/` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:24:48.008064 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 440/859] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jul 25 05:24:48 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:24:48.008589 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbab33e4-8491-4d94-9c41-bf3b518ee373 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:24:48.014940 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 420/860] 10.222.0.33 () {56 vars in 1148 bytes} [Mon Jul 25 05:24:47 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 05:24:48.016638 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed0e5fe-f71f-4780-b6c8-d48c3d36bdbb None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:24:48.017180 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed0e5fe-f71f-4780-b6c8-d48c3d36bdbb None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:24:48.017782 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed0e5fe-f71f-4780-b6c8-d48c3d36bdbb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:24:48.035170 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-2ed0e5fe-f71f-4780-b6c8-d48c3d36bdbb None None] MFA Rules not processed for user `1eae540242d94c10b0ad33960a855f08`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:24:48.036368 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-2ed0e5fe-f71f-4780-b6c8-d48c3d36bdbb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:24:48.107376 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 421/861] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jul 25 05:24:48 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 25 05:24:48.109014 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed0e5fe-f71f-4780-b6c8-d48c3d36bdbb None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:24:48.121159 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbab33e4-8491-4d94-9c41-bf3b518ee373 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:24:48.159829 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb9ec94-1860-489b-9f83-28e4389ab82e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:24:48.160768 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb9ec94-1860-489b-9f83-28e4389ab82e None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:24:48.161083 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb9ec94-1860-489b-9f83-28e4389ab82e None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:24:48.161365 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb9ec94-1860-489b-9f83-28e4389ab82e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:24:48.172656 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebb9ec94-1860-489b-9f83-28e4389ab82e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:24:48.174371 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebb9ec94-1860-489b-9f83-28e4389ab82e None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:24:48.180162 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:24:48.194888 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application [None req-ebb9ec94-1860-489b-9f83-28e4389ab82e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 05:24:48.194888 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:24:48.194888 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:24:48.194888 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:24:48.194888 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:24:48.194888 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:24:48.194888 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:24:48.194888 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:24:48.194888 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:24:48.194888 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:24:48.194888 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:24:48.194888 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:24:48.194888 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 05:24:48.194888 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 05:24:48.194888 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 05:24:48.195947 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 05:24:48.195947 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:24:48.195947 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:24:48.195947 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:24:48.195947 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:24:48.195947 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:24:48.195947 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:24:48.195947 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:24:48.195947 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application with Lock( Jul 25 05:24:48.195947 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:24:48.195947 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:24:48.195947 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:24:48.195947 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:24:48.195947 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:24:48.195947 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:24:48.195947 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:24:48.195947 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:24:48.196879 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 05:24:48.196879 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 05:24:48.196879 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 05:24:48.196879 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Jul 25 05:24:48.196879 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 441/862] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Jul 25 05:24:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:24:48.197173 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb9ec94-1860-489b-9f83-28e4389ab82e None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:24:48.223581 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed0e5fe-f71f-4780-b6c8-d48c3d36bdbb None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:24:48.242042 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:24:48.262225 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e7e83df-ce7b-4690-bd60-827a2ce06c91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:24:48.262883 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7e83df-ce7b-4690-bd60-827a2ce06c91 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:24:48.263168 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7e83df-ce7b-4690-bd60-827a2ce06c91 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:24:48.263447 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7e83df-ce7b-4690-bd60-827a2ce06c91 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:24:48.264283 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e7e83df-ce7b-4690-bd60-827a2ce06c91 None admin] RBAC: Adding query filter params (role.id=200e95cafbc54269a51756ddd406082f, scope.project.id=92afba266cce4d1bad4134c351e74be3, user.id=3352531be1b54f76b864383ccc2e5ec2) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:24:48.264597 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e7e83df-ce7b-4690-bd60-827a2ce06c91 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:24:48.266142 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e7e83df-ce7b-4690-bd60-827a2ce06c91 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:24:48.272052 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 422/863] 10.222.0.33 () {56 vars in 1374 bytes} [Mon Jul 25 05:24:48 2022] GET /identity/v3/role_assignments?user.id=3352531be1b54f76b864383ccc2e5ec2&scope.project.id=92afba266cce4d1bad4134c351e74be3&role.id=200e95cafbc54269a51756ddd406082f => generated 608 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:24:48.272837 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e7e83df-ce7b-4690-bd60-827a2ce06c91 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:24:48.335469 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb9ec94-1860-489b-9f83-28e4389ab82e None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:24:48.386849 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46815a6b-c8dc-462c-9b27-5598f0846f75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:24:48.388694 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46815a6b-c8dc-462c-9b27-5598f0846f75 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:24:48.389194 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46815a6b-c8dc-462c-9b27-5598f0846f75 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:24:48.389897 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46815a6b-c8dc-462c-9b27-5598f0846f75 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:24:48.391161 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46815a6b-c8dc-462c-9b27-5598f0846f75 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:24:48.393048 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46815a6b-c8dc-462c-9b27-5598f0846f75 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:24:48.393738 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e7e83df-ce7b-4690-bd60-827a2ce06c91 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:24:48.414706 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application [None req-46815a6b-c8dc-462c-9b27-5598f0846f75 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 25 05:24:48.414706 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:24:48.414706 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 25 05:24:48.414706 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 25 05:24:48.414706 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 25 05:24:48.414706 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 25 05:24:48.414706 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 25 05:24:48.414706 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 25 05:24:48.414706 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 25 05:24:48.414706 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Jul 25 05:24:48.414706 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 25 05:24:48.414706 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Jul 25 05:24:48.414706 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:24:48.414706 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:24:48.414706 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:24:48.414706 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:24:48.415731 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:24:48.415731 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:24:48.415731 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:24:48.415731 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:24:48.415731 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:24:48.415731 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:24:48.415731 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:24:48.415731 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 25 05:24:48.415731 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 25 05:24:48.415731 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 25 05:24:48.415731 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 25 05:24:48.415731 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:24:48.415731 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:24:48.415731 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:24:48.415731 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:24:48.415731 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:24:48.416571 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:24:48.416571 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:24:48.416571 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application with Lock( Jul 25 05:24:48.416571 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:24:48.416571 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:24:48.416571 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:24:48.416571 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:24:48.416571 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:24:48.416571 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:24:48.416571 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:24:48.416571 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:24:48.416571 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 25 05:24:48.416571 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 25 05:24:48.416571 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 25 05:24:48.416571 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Jul 25 05:24:48.419548 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 442/864] 10.222.0.33 () {56 vars in 1132 bytes} [Mon Jul 25 05:24:48 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:24:48.424613 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46815a6b-c8dc-462c-9b27-5598f0846f75 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:24:48.431714 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:24:48.447370 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f824deb-5bbf-4222-b3b7-b0824f7e249f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:24:48.448103 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f824deb-5bbf-4222-b3b7-b0824f7e249f None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:24:48.448914 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f824deb-5bbf-4222-b3b7-b0824f7e249f None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:24:48.449301 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f824deb-5bbf-4222-b3b7-b0824f7e249f None admin] PATH_INFO: `/v3/domains` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:24:48.450023 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f824deb-5bbf-4222-b3b7-b0824f7e249f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:24:48.450336 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f824deb-5bbf-4222-b3b7-b0824f7e249f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:24:48.451789 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f824deb-5bbf-4222-b3b7-b0824f7e249f None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:24:48.466816 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 423/865] 10.222.0.33 () {56 vars in 1109 bytes} [Mon Jul 25 05:24:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:24:48.472640 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f824deb-5bbf-4222-b3b7-b0824f7e249f None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:24:48.562508 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46815a6b-c8dc-462c-9b27-5598f0846f75 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:24:48.564587 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:24:48.603650 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f824deb-5bbf-4222-b3b7-b0824f7e249f None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:24:48.607891 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa9038d9-1e98-4af7-a2a6-2eb8e1a015a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:24:48.608617 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9038d9-1e98-4af7-a2a6-2eb8e1a015a1 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:24:48.608934 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9038d9-1e98-4af7-a2a6-2eb8e1a015a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:24:48.609215 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9038d9-1e98-4af7-a2a6-2eb8e1a015a1 None admin] PATH_INFO: `/v3/services` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:24:48.609932 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa9038d9-1e98-4af7-a2a6-2eb8e1a015a1 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:24:48.610239 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa9038d9-1e98-4af7-a2a6-2eb8e1a015a1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:24:48.612139 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa9038d9-1e98-4af7-a2a6-2eb8e1a015a1 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:24:48.618892 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 443/866] 10.222.0.33 () {56 vars in 1126 bytes} [Mon Jul 25 05:24:48 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 05:24:48.624914 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa9038d9-1e98-4af7-a2a6-2eb8e1a015a1 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:24:48.641775 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e43d55f-cfd8-4548-8fbf-69eb6702da24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:24:48.642534 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e43d55f-cfd8-4548-8fbf-69eb6702da24 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:24:48.642835 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e43d55f-cfd8-4548-8fbf-69eb6702da24 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:24:48.643116 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e43d55f-cfd8-4548-8fbf-69eb6702da24 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:24:48.650779 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e43d55f-cfd8-4548-8fbf-69eb6702da24 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:24:48.652571 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e43d55f-cfd8-4548-8fbf-69eb6702da24 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:24:48.668170 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application [None req-0e43d55f-cfd8-4548-8fbf-69eb6702da24 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 05:24:48.668170 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:24:48.668170 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:24:48.668170 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:24:48.668170 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:24:48.668170 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:24:48.668170 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:24:48.668170 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:24:48.668170 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:24:48.668170 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:24:48.668170 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:24:48.668170 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:24:48.668170 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 25 05:24:48.668170 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 25 05:24:48.668170 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 25 05:24:48.669172 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 25 05:24:48.669172 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:24:48.669172 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:24:48.669172 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:24:48.669172 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:24:48.669172 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:24:48.669172 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:24:48.669172 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:24:48.669172 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application with Lock( Jul 25 05:24:48.669172 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:24:48.669172 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:24:48.669172 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:24:48.669172 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:24:48.669172 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:24:48.669172 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:24:48.669172 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:24:48.669172 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:24:48.670316 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 25 05:24:48.670316 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 25 05:24:48.670316 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 25 05:24:48.670316 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 25 05:24:48.670316 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 25 05:24:48.670316 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 25 05:24:48.670316 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 05:24:48.670316 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Jul 25 05:24:48.670316 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 424/867] 10.222.0.33 () {56 vars in 1111 bytes} [Mon Jul 25 05:24:48 2022] GET /identity/v3/projects/service => generated 88 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:24:48.676530 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e43d55f-cfd8-4548-8fbf-69eb6702da24 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:24:48.734558 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:24:48.754004 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa9038d9-1e98-4af7-a2a6-2eb8e1a015a1 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:24:48.799073 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e43d55f-cfd8-4548-8fbf-69eb6702da24 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:24:48.804507 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ec26e7-3445-4a4f-a069-54f96d292bbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:24:48.805225 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ec26e7-3445-4a4f-a069-54f96d292bbf None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:24:48.805519 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ec26e7-3445-4a4f-a069-54f96d292bbf None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:24:48.805801 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ec26e7-3445-4a4f-a069-54f96d292bbf None admin] PATH_INFO: `/v3/services` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:24:48.806527 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13ec26e7-3445-4a4f-a069-54f96d292bbf None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:24:48.806840 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13ec26e7-3445-4a4f-a069-54f96d292bbf None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:24:48.809628 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13ec26e7-3445-4a4f-a069-54f96d292bbf None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:24:48.818052 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 444/868] 10.222.0.33 () {56 vars in 1126 bytes} [Mon Jul 25 05:24:48 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:24:48.857207 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c26517c-2ad6-4f2d-b8cf-e24a489cbe7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:24:48.857889 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c26517c-2ad6-4f2d-b8cf-e24a489cbe7a None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:24:48.858171 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c26517c-2ad6-4f2d-b8cf-e24a489cbe7a None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:24:48.858449 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c26517c-2ad6-4f2d-b8cf-e24a489cbe7a None admin] PATH_INFO: `/v3/projects` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:24:48.859252 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c26517c-2ad6-4f2d-b8cf-e24a489cbe7a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:24:48.859560 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c26517c-2ad6-4f2d-b8cf-e24a489cbe7a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:24:48.861025 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c26517c-2ad6-4f2d-b8cf-e24a489cbe7a None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:24:48.875845 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 425/869] 10.222.0.33 () {56 vars in 1148 bytes} [Mon Jul 25 05:24:48 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 05:24:48.882034 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ec26e7-3445-4a4f-a069-54f96d292bbf None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:24:48.921185 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:24:49.001273 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ec26e7-3445-4a4f-a069-54f96d292bbf None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:24:49.037692 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-417ee440-248e-4f6e-9972-c9ffa63c8efd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:24:49.038361 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417ee440-248e-4f6e-9972-c9ffa63c8efd None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:24:49.038646 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417ee440-248e-4f6e-9972-c9ffa63c8efd None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:24:49.038927 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417ee440-248e-4f6e-9972-c9ffa63c8efd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:24:49.039775 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-417ee440-248e-4f6e-9972-c9ffa63c8efd None admin] RBAC: Adding query filter params (role.id=39e5c666285a42fe8fc2c3afca6260d1, scope.project.id=92afba266cce4d1bad4134c351e74be3, user.id=eb135a8d5d7444cfbfa336115ba2ba44) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:24:49.040090 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-417ee440-248e-4f6e-9972-c9ffa63c8efd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:24:49.041562 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-417ee440-248e-4f6e-9972-c9ffa63c8efd None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:24:49.048184 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 445/870] 10.222.0.33 () {56 vars in 1374 bytes} [Mon Jul 25 05:24:48 2022] GET /identity/v3/role_assignments?user.id=eb135a8d5d7444cfbfa336115ba2ba44&scope.project.id=92afba266cce4d1bad4134c351e74be3&role.id=39e5c666285a42fe8fc2c3afca6260d1 => generated 608 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:24:49.483059 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca2d9d3-182f-4c25-a081-d067072691b1 None None] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:24:49.483597 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca2d9d3-182f-4c25-a081-d067072691b1 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:24:49.483597 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca2d9d3-182f-4c25-a081-d067072691b1 None None] PATH_INFO: `/` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:24:49.485262 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 426/871] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jul 25 05:24:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:24:49.490634 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ecd607e-1517-460e-a32e-16dfa4655daa None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:24:49.491046 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ecd607e-1517-460e-a32e-16dfa4655daa None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:24:49.491235 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ecd607e-1517-460e-a32e-16dfa4655daa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:24:49.509236 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-7ecd607e-1517-460e-a32e-16dfa4655daa None None] MFA Rules not processed for user `1eae540242d94c10b0ad33960a855f08`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:24:49.510430 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-7ecd607e-1517-460e-a32e-16dfa4655daa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:24:49.581339 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 446/872] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jul 25 05:24:49 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 25 05:24:49.605898 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca2d9d3-182f-4c25-a081-d067072691b1 None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:24:49.713862 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca2d9d3-182f-4c25-a081-d067072691b1 None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:24:49.723741 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:24:49.750549 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eefde42-3b5d-46d5-84ad-232338219bc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:24:49.751458 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eefde42-3b5d-46d5-84ad-232338219bc9 None admin] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:24:49.751998 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eefde42-3b5d-46d5-84ad-232338219bc9 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:24:49.752428 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eefde42-3b5d-46d5-84ad-232338219bc9 None admin] PATH_INFO: `/v3/services` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:24:49.753545 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eefde42-3b5d-46d5-84ad-232338219bc9 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:24:49.755048 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eefde42-3b5d-46d5-84ad-232338219bc9 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:24:49.772350 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 427/873] 10.222.0.33 () {60 vars in 1141 bytes} [Mon Jul 25 05:24:49 2022] POST /identity/v3/services => generated 251 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 05:24:50.583863 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937fd270-2454-4fa8-b8b8-b819a9b743cf None None] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:24:50.584403 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937fd270-2454-4fa8-b8b8-b819a9b743cf None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:24:50.584715 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937fd270-2454-4fa8-b8b8-b819a9b743cf None None] PATH_INFO: `/` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:24:50.586581 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 447/874] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jul 25 05:24:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:24:50.592857 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4713f08-304e-461f-b321-b0462207c582 None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:24:50.593275 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4713f08-304e-461f-b321-b0462207c582 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:24:50.593621 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4713f08-304e-461f-b321-b0462207c582 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:24:50.610727 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-b4713f08-304e-461f-b321-b0462207c582 None None] MFA Rules not processed for user `1eae540242d94c10b0ad33960a855f08`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:24:50.611929 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-b4713f08-304e-461f-b321-b0462207c582 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:24:50.692892 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 428/875] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jul 25 05:24:50 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 25 05:24:50.712087 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-937fd270-2454-4fa8-b8b8-b819a9b743cf None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:24:50.737284 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:24:50.825081 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-937fd270-2454-4fa8-b8b8-b819a9b743cf None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:24:50.866912 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0972c79b-46b9-4474-a447-0e7ff08da24b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:24:50.867563 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0972c79b-46b9-4474-a447-0e7ff08da24b None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:24:50.867864 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0972c79b-46b9-4474-a447-0e7ff08da24b None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:24:50.869814 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0972c79b-46b9-4474-a447-0e7ff08da24b None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:24:50.879822 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0972c79b-46b9-4474-a447-0e7ff08da24b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 05:24:50.888623 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application [None req-0972c79b-46b9-4474-a447-0e7ff08da24b None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 25 05:24:50.888623 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:24:50.888623 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:24:50.888623 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:24:50.888623 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:24:50.888623 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:24:50.888623 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:24:50.888623 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:24:50.888623 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:24:50.888623 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:24:50.888623 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:24:50.888623 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:24:50.888623 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 05:24:50.888623 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 05:24:50.888623 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 05:24:50.888623 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 05:24:50.889596 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 05:24:50.889596 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 05:24:50.889596 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 05:24:50.889596 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 05:24:50.889596 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:24:50.889596 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:24:50.889596 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:24:50.889596 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:24:50.889596 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:24:50.889596 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:24:50.889596 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:24:50.889596 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application with Lock( Jul 25 05:24:50.889596 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:24:50.889596 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:24:50.889596 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:24:50.889596 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:24:50.890435 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:24:50.890435 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:24:50.890435 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:24:50.890435 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:24:50.890435 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 05:24:50.890435 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 05:24:50.890435 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 05:24:50.890435 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 05:24:50.890435 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 05:24:50.890435 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 05:24:50.890435 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 25 05:24:50.890435 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Jul 25 05:24:50.890435 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 448/876] 10.222.0.33 () {56 vars in 1120 bytes} [Mon Jul 25 05:24:50 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:24:50.895995 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4713f08-304e-461f-b321-b0462207c582 None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:24:50.933400 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29db13bd-b8a7-4819-bdf6-553b3802e9ac None None] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:24:50.933737 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29db13bd-b8a7-4819-bdf6-553b3802e9ac None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:24:50.933971 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29db13bd-b8a7-4819-bdf6-553b3802e9ac None None] PATH_INFO: `/` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:24:50.936632 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 449/877] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jul 25 05:24:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:24:50.942495 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a1cef9-2e43-4c2f-abd6-cda006f07331 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:24:50.942810 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a1cef9-2e43-4c2f-abd6-cda006f07331 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:24:50.943059 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a1cef9-2e43-4c2f-abd6-cda006f07331 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:24:50.967530 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-f9a1cef9-2e43-4c2f-abd6-cda006f07331 None None] MFA Rules not processed for user `1eae540242d94c10b0ad33960a855f08`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:24:50.969065 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-f9a1cef9-2e43-4c2f-abd6-cda006f07331 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:24:51.007584 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:24:51.018816 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4713f08-304e-461f-b321-b0462207c582 None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:24:51.055937 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-255cf969-da29-4a77-ab02-8d08424a96dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:24:51.057155 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255cf969-da29-4a77-ab02-8d08424a96dd None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:24:51.058344 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255cf969-da29-4a77-ab02-8d08424a96dd None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:24:51.058860 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255cf969-da29-4a77-ab02-8d08424a96dd None admin] PATH_INFO: `/v3/roles` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:24:51.059809 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-255cf969-da29-4a77-ab02-8d08424a96dd None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:24:51.060309 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-255cf969-da29-4a77-ab02-8d08424a96dd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:24:51.062129 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-255cf969-da29-4a77-ab02-8d08424a96dd None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:24:51.064869 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 450/878] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jul 25 05:24:50 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 25 05:24:51.076427 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 429/879] 10.222.0.33 () {56 vars in 1115 bytes} [Mon Jul 25 05:24:50 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:24:51.084052 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a1cef9-2e43-4c2f-abd6-cda006f07331 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:24:51.090293 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-255cf969-da29-4a77-ab02-8d08424a96dd None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:24:51.103841 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:24:51.190755 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a1cef9-2e43-4c2f-abd6-cda006f07331 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:24:51.212374 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-255cf969-da29-4a77-ab02-8d08424a96dd None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:24:51.227884 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a29f3d20-e588-41a0-b694-1651950042a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:24:51.228759 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29f3d20-e588-41a0-b694-1651950042a2 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:24:51.229149 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29f3d20-e588-41a0-b694-1651950042a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:24:51.230037 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29f3d20-e588-41a0-b694-1651950042a2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:24:51.241012 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a29f3d20-e588-41a0-b694-1651950042a2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:24:51.242467 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a29f3d20-e588-41a0-b694-1651950042a2 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:24:51.249032 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dff3048-68bc-4cf3-8aa2-0de090801429 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:24:51.249703 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dff3048-68bc-4cf3-8aa2-0de090801429 None admin] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:24:51.249992 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dff3048-68bc-4cf3-8aa2-0de090801429 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:24:51.250496 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dff3048-68bc-4cf3-8aa2-0de090801429 None admin] PATH_INFO: `/v3/services` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:24:51.250991 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dff3048-68bc-4cf3-8aa2-0de090801429 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:24:51.252340 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dff3048-68bc-4cf3-8aa2-0de090801429 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:24:51.261224 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application [None req-a29f3d20-e588-41a0-b694-1651950042a2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 05:24:51.261224 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:24:51.261224 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:24:51.261224 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:24:51.261224 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:24:51.261224 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:24:51.261224 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:24:51.261224 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:24:51.261224 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:24:51.261224 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:24:51.261224 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:24:51.261224 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:24:51.261224 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 05:24:51.261224 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 05:24:51.261224 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 05:24:51.262153 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 05:24:51.262153 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:24:51.262153 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:24:51.262153 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:24:51.262153 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:24:51.262153 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:24:51.262153 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:24:51.262153 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:24:51.262153 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application with Lock( Jul 25 05:24:51.262153 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:24:51.262153 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:24:51.262153 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:24:51.262153 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:24:51.262153 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:24:51.262153 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:24:51.262153 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:24:51.262153 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:24:51.263076 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 05:24:51.263076 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 05:24:51.263076 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 05:24:51.263076 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Jul 25 05:24:51.263076 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 451/880] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Jul 25 05:24:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:24:51.264011 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99638091-14e3-43e1-809c-de263927b91f None None] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:24:51.264011 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99638091-14e3-43e1-809c-de263927b91f None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:24:51.264282 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99638091-14e3-43e1-809c-de263927b91f None None] PATH_INFO: `/` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:24:51.265580 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 452/881] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jul 25 05:24:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:24:51.269344 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 430/882] 10.222.0.33 () {60 vars in 1141 bytes} [Mon Jul 25 05:24:51 2022] POST /identity/v3/services => generated 269 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 05:24:51.271835 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99638091-14e3-43e1-809c-de263927b91f None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:24:51.272563 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7397d0a-4e8e-4f5e-a697-0fb65b595103 None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:24:51.273126 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7397d0a-4e8e-4f5e-a697-0fb65b595103 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:24:51.273556 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7397d0a-4e8e-4f5e-a697-0fb65b595103 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:24:51.293288 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-e7397d0a-4e8e-4f5e-a697-0fb65b595103 None None] MFA Rules not processed for user `1eae540242d94c10b0ad33960a855f08`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:24:51.294838 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-e7397d0a-4e8e-4f5e-a697-0fb65b595103 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:24:51.308729 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:24:51.365231 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:24:51.383690 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 431/883] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jul 25 05:24:51 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 25 05:24:51.406428 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7397d0a-4e8e-4f5e-a697-0fb65b595103 None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:24:51.411036 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99638091-14e3-43e1-809c-de263927b91f None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:24:51.455577 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ff1a0e-d7c0-4ed4-a489-c3d855685f93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:24:51.456173 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ff1a0e-d7c0-4ed4-a489-c3d855685f93 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:24:51.456470 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ff1a0e-d7c0-4ed4-a489-c3d855685f93 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:24:51.456748 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ff1a0e-d7c0-4ed4-a489-c3d855685f93 None admin] PATH_INFO: `/v3/domains` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:24:51.457546 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70ff1a0e-d7c0-4ed4-a489-c3d855685f93 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:24:51.457879 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70ff1a0e-d7c0-4ed4-a489-c3d855685f93 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:24:51.459240 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70ff1a0e-d7c0-4ed4-a489-c3d855685f93 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:24:51.472213 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 453/884] 10.222.0.33 () {56 vars in 1109 bytes} [Mon Jul 25 05:24:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:24:51.477547 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ff1a0e-d7c0-4ed4-a489-c3d855685f93 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:24:51.525606 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7397d0a-4e8e-4f5e-a697-0fb65b595103 None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:24:51.564698 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d716d0d1-0913-4199-b11f-f7b07e70eff8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:24:51.565555 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d716d0d1-0913-4199-b11f-f7b07e70eff8 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:24:51.565839 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d716d0d1-0913-4199-b11f-f7b07e70eff8 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:24:51.566114 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d716d0d1-0913-4199-b11f-f7b07e70eff8 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:24:51.566798 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d716d0d1-0913-4199-b11f-f7b07e70eff8 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:24:51.568155 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d716d0d1-0913-4199-b11f-f7b07e70eff8 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:24:51.581843 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application [None req-d716d0d1-0913-4199-b11f-f7b07e70eff8 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 25 05:24:51.581843 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:24:51.581843 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 25 05:24:51.581843 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 25 05:24:51.581843 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 25 05:24:51.581843 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 25 05:24:51.581843 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 25 05:24:51.581843 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 25 05:24:51.581843 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 25 05:24:51.581843 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Jul 25 05:24:51.581843 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 25 05:24:51.581843 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Jul 25 05:24:51.581843 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:24:51.581843 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:24:51.581843 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:24:51.581843 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:24:51.581843 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:24:51.582900 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:24:51.582900 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:24:51.582900 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:24:51.582900 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:24:51.582900 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:24:51.582900 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:24:51.582900 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 25 05:24:51.582900 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 25 05:24:51.582900 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 25 05:24:51.582900 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 25 05:24:51.582900 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:24:51.582900 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:24:51.582900 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:24:51.582900 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:24:51.582900 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:24:51.582900 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:24:51.583761 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:24:51.583761 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application with Lock( Jul 25 05:24:51.583761 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:24:51.583761 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:24:51.583761 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:24:51.583761 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:24:51.583761 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:24:51.583761 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:24:51.583761 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:24:51.583761 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:24:51.583761 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 25 05:24:51.583761 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 25 05:24:51.583761 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 25 05:24:51.583761 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Jul 25 05:24:51.583761 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 432/885] 10.222.0.33 () {56 vars in 1126 bytes} [Mon Jul 25 05:24:51 2022] GET /identity/v3/services/object-store => generated 93 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 25 05:24:51.588998 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d716d0d1-0913-4199-b11f-f7b07e70eff8 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:24:51.605238 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:24:51.609081 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ff1a0e-d7c0-4ed4-a489-c3d855685f93 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:24:51.637813 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:24:51.648049 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-988e81b2-2b7b-4e1c-9a0b-0c3174393864 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:24:51.649041 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988e81b2-2b7b-4e1c-9a0b-0c3174393864 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:24:51.649431 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988e81b2-2b7b-4e1c-9a0b-0c3174393864 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:24:51.649816 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988e81b2-2b7b-4e1c-9a0b-0c3174393864 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:24:51.659672 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-988e81b2-2b7b-4e1c-9a0b-0c3174393864 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:24:51.661104 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-988e81b2-2b7b-4e1c-9a0b-0c3174393864 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:24:51.677118 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application [None req-988e81b2-2b7b-4e1c-9a0b-0c3174393864 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 25 05:24:51.677118 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:24:51.677118 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:24:51.677118 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:24:51.677118 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:24:51.677118 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:24:51.677118 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:24:51.677118 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:24:51.677118 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:24:51.677118 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:24:51.677118 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:24:51.677118 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:24:51.677118 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 25 05:24:51.677118 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 25 05:24:51.677118 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 25 05:24:51.677118 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 25 05:24:51.678367 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:24:51.678367 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:24:51.678367 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 25 05:24:51.678367 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 05:24:51.678367 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 25 05:24:51.678367 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 05:24:51.678367 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:24:51.678367 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:24:51.678367 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:24:51.678367 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:24:51.678367 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:24:51.678367 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application with Lock( Jul 25 05:24:51.678367 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:24:51.678367 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:24:51.678367 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:24:51.678367 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:24:51.678367 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:24:51.678367 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:24:51.679543 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:24:51.679543 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:24:51.679543 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 25 05:24:51.679543 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 25 05:24:51.679543 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 25 05:24:51.679543 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 25 05:24:51.679543 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 25 05:24:51.679543 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 25 05:24:51.679543 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 25 05:24:51.679543 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Jul 25 05:24:51.682272 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 454/886] 10.222.0.33 () {56 vars in 1117 bytes} [Mon Jul 25 05:24:51 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 25 05:24:51.687507 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-988e81b2-2b7b-4e1c-9a0b-0c3174393864 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:24:51.717634 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d716d0d1-0913-4199-b11f-f7b07e70eff8 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:24:51.754894 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc45c252-ef12-491b-a8d5-0c55b81b9291 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:24:51.755913 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc45c252-ef12-491b-a8d5-0c55b81b9291 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:24:51.756419 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc45c252-ef12-491b-a8d5-0c55b81b9291 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:24:51.757009 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc45c252-ef12-491b-a8d5-0c55b81b9291 None admin] PATH_INFO: `/v3/services` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:24:51.758050 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc45c252-ef12-491b-a8d5-0c55b81b9291 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:24:51.758644 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc45c252-ef12-491b-a8d5-0c55b81b9291 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:24:51.760202 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc45c252-ef12-491b-a8d5-0c55b81b9291 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:24:51.766201 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 433/887] 10.222.0.33 () {56 vars in 1122 bytes} [Mon Jul 25 05:24:51 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:24:51.773755 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc45c252-ef12-491b-a8d5-0c55b81b9291 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:24:51.811707 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-988e81b2-2b7b-4e1c-9a0b-0c3174393864 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:24:51.859759 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da351fe-69b6-4129-8a36-6247f0d93778 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:24:51.860773 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da351fe-69b6-4129-8a36-6247f0d93778 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:24:51.861276 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da351fe-69b6-4129-8a36-6247f0d93778 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:24:51.861726 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da351fe-69b6-4129-8a36-6247f0d93778 None admin] PATH_INFO: `/v3/users` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:24:51.862754 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7da351fe-69b6-4129-8a36-6247f0d93778 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:24:51.863240 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7da351fe-69b6-4129-8a36-6247f0d93778 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:24:51.865311 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7da351fe-69b6-4129-8a36-6247f0d93778 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:24:51.869799 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:24:51.889577 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 455/888] 10.222.0.33 () {56 vars in 1149 bytes} [Mon Jul 25 05:24:51 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:24:51.894607 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da351fe-69b6-4129-8a36-6247f0d93778 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:24:51.896853 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc45c252-ef12-491b-a8d5-0c55b81b9291 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:24:51.936083 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4801c3d4-8637-4807-9dd3-329286594f34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:24:51.936756 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4801c3d4-8637-4807-9dd3-329286594f34 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:24:51.937089 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4801c3d4-8637-4807-9dd3-329286594f34 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:24:51.937372 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4801c3d4-8637-4807-9dd3-329286594f34 None admin] PATH_INFO: `/v3/services` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:24:51.938551 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4801c3d4-8637-4807-9dd3-329286594f34 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:24:51.939154 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4801c3d4-8637-4807-9dd3-329286594f34 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:24:51.940890 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4801c3d4-8637-4807-9dd3-329286594f34 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:24:51.943715 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:24:51.952035 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 434/889] 10.222.0.33 () {56 vars in 1122 bytes} [Mon Jul 25 05:24:51 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:24:51.952035 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0988091-9cbc-4970-b7c1-bb9d1792b089 None None] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:24:51.952035 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0988091-9cbc-4970-b7c1-bb9d1792b089 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:24:51.952035 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0988091-9cbc-4970-b7c1-bb9d1792b089 None None] PATH_INFO: `/` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:24:51.952035 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 435/890] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jul 25 05:24:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:24:51.956183 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5883720-acf3-4745-adcd-a9045378d08b None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:24:51.956471 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5883720-acf3-4745-adcd-a9045378d08b None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:24:51.956749 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5883720-acf3-4745-adcd-a9045378d08b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:24:51.977458 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-f5883720-acf3-4745-adcd-a9045378d08b None None] MFA Rules not processed for user `1eae540242d94c10b0ad33960a855f08`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:24:51.978612 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-f5883720-acf3-4745-adcd-a9045378d08b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:24:52.021918 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da351fe-69b6-4129-8a36-6247f0d93778 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:24:52.063700 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 436/891] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jul 25 05:24:51 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 25 05:24:52.069482 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e5e2ac-cd04-4477-bf38-e656ab266be8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:24:52.070419 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e5e2ac-cd04-4477-bf38-e656ab266be8 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:24:52.070850 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e5e2ac-cd04-4477-bf38-e656ab266be8 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:24:52.071272 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e5e2ac-cd04-4477-bf38-e656ab266be8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:24:52.083060 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5883720-acf3-4745-adcd-a9045378d08b None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:24:52.156569 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48e5e2ac-cd04-4477-bf38-e656ab266be8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:24:52.157807 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:24:52.159243 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48e5e2ac-cd04-4477-bf38-e656ab266be8 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:24:52.208122 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:24:52.220803 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5883720-acf3-4745-adcd-a9045378d08b None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:24:52.262295 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e02ec9e-cc31-49ff-810a-67d2c0b1a6e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:24:52.263167 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e02ec9e-cc31-49ff-810a-67d2c0b1a6e8 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:24:52.263590 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e02ec9e-cc31-49ff-810a-67d2c0b1a6e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:24:52.263873 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e02ec9e-cc31-49ff-810a-67d2c0b1a6e8 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:24:52.264806 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e02ec9e-cc31-49ff-810a-67d2c0b1a6e8 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:24:52.266930 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e02ec9e-cc31-49ff-810a-67d2c0b1a6e8 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:24:52.277412 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 456/892] 10.222.0.33 () {58 vars in 1282 bytes} [Mon Jul 25 05:24:51 2022] GET /identity/v3/auth/tokens => generated 1362 bytes in 383 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 25 05:24:52.281584 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application [None req-5e02ec9e-cc31-49ff-810a-67d2c0b1a6e8 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 25 05:24:52.281584 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:24:52.281584 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 25 05:24:52.281584 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 25 05:24:52.281584 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 25 05:24:52.281584 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 25 05:24:52.281584 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 25 05:24:52.281584 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 25 05:24:52.281584 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 25 05:24:52.281584 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Jul 25 05:24:52.281584 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 25 05:24:52.281584 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Jul 25 05:24:52.281584 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:24:52.281584 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:24:52.281584 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:24:52.281584 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:24:52.281584 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:24:52.282562 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:24:52.282562 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:24:52.282562 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:24:52.282562 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:24:52.282562 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:24:52.282562 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:24:52.282562 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 25 05:24:52.282562 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 25 05:24:52.282562 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 25 05:24:52.282562 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 25 05:24:52.282562 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:24:52.282562 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:24:52.282562 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:24:52.282562 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:24:52.282562 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:24:52.282562 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:24:52.283415 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:24:52.283415 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application with Lock( Jul 25 05:24:52.283415 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:24:52.283415 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:24:52.283415 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:24:52.283415 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:24:52.283415 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:24:52.283415 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:24:52.283415 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:24:52.283415 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:24:52.283415 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 25 05:24:52.283415 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 25 05:24:52.283415 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 25 05:24:52.283415 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Jul 25 05:24:52.284163 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e5e2ac-cd04-4477-bf38-e656ab266be8 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:24:52.284235 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 437/893] 10.222.0.33 () {56 vars in 1114 bytes} [Mon Jul 25 05:24:52 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:24:52.290001 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e02ec9e-cc31-49ff-810a-67d2c0b1a6e8 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:24:52.399108 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e5e2ac-cd04-4477-bf38-e656ab266be8 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:24:52.407252 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e02ec9e-cc31-49ff-810a-67d2c0b1a6e8 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:24:52.435933 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:24:52.445945 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4554feca-85b1-4892-96b4-e250625bec22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:24:52.446507 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72bb348d-68a7-48f1-abf0-0e21cc4ede49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:24:52.446922 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4554feca-85b1-4892-96b4-e250625bec22 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:24:52.447305 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4554feca-85b1-4892-96b4-e250625bec22 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:24:52.447649 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4554feca-85b1-4892-96b4-e250625bec22 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:24:52.448004 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72bb348d-68a7-48f1-abf0-0e21cc4ede49 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:24:52.449113 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72bb348d-68a7-48f1-abf0-0e21cc4ede49 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:24:52.449603 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72bb348d-68a7-48f1-abf0-0e21cc4ede49 None admin] PATH_INFO: `/v3/services` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:24:52.450503 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72bb348d-68a7-48f1-abf0-0e21cc4ede49 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:24:52.451023 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72bb348d-68a7-48f1-abf0-0e21cc4ede49 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:24:52.452606 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72bb348d-68a7-48f1-abf0-0e21cc4ede49 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:24:52.458742 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 438/894] 10.222.0.33 () {56 vars in 1114 bytes} [Mon Jul 25 05:24:52 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:24:52.460492 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4554feca-85b1-4892-96b4-e250625bec22 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:24:52.468982 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4554feca-85b1-4892-96b4-e250625bec22 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:24:52.469146 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72bb348d-68a7-48f1-abf0-0e21cc4ede49 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:24:52.469925 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:24:52.492814 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application [None req-4554feca-85b1-4892-96b4-e250625bec22 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 05:24:52.492814 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:24:52.492814 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:24:52.492814 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:24:52.492814 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:24:52.492814 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:24:52.492814 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:24:52.492814 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:24:52.492814 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:24:52.492814 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:24:52.492814 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:24:52.492814 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:24:52.492814 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 05:24:52.492814 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 05:24:52.492814 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 05:24:52.493891 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 05:24:52.493891 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:24:52.493891 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:24:52.493891 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:24:52.493891 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:24:52.493891 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:24:52.493891 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:24:52.493891 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:24:52.493891 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application with Lock( Jul 25 05:24:52.493891 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:24:52.493891 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:24:52.493891 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:24:52.493891 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:24:52.493891 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:24:52.493891 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:24:52.493891 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:24:52.493891 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:24:52.495069 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 05:24:52.495069 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 05:24:52.495069 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 05:24:52.495069 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Jul 25 05:24:52.495069 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 457/895] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Jul 25 05:24:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:24:52.501522 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4554feca-85b1-4892-96b4-e250625bec22 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:24:52.590376 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72bb348d-68a7-48f1-abf0-0e21cc4ede49 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:24:52.621965 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4554feca-85b1-4892-96b4-e250625bec22 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:24:52.635399 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3935534e-a958-44cf-90a3-8a1ff4f12e38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:24:52.637272 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3935534e-a958-44cf-90a3-8a1ff4f12e38 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:24:52.637700 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3935534e-a958-44cf-90a3-8a1ff4f12e38 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:24:52.638139 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3935534e-a958-44cf-90a3-8a1ff4f12e38 None admin] PATH_INFO: `/v3/services` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:24:52.638993 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3935534e-a958-44cf-90a3-8a1ff4f12e38 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:24:52.639444 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3935534e-a958-44cf-90a3-8a1ff4f12e38 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:24:52.641133 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3935534e-a958-44cf-90a3-8a1ff4f12e38 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:24:52.652225 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 439/896] 10.222.0.33 () {56 vars in 1114 bytes} [Mon Jul 25 05:24:52 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 05:24:52.653526 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3935534e-a958-44cf-90a3-8a1ff4f12e38 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:24:52.658865 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c34d62-c780-4209-8a2c-7ce02ed80334 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:24:52.659541 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c34d62-c780-4209-8a2c-7ce02ed80334 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:24:52.659832 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c34d62-c780-4209-8a2c-7ce02ed80334 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:24:52.660117 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c34d62-c780-4209-8a2c-7ce02ed80334 None admin] PATH_INFO: `/v3/domains` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:24:52.660879 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4c34d62-c780-4209-8a2c-7ce02ed80334 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:24:52.661195 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4c34d62-c780-4209-8a2c-7ce02ed80334 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:24:52.662570 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4c34d62-c780-4209-8a2c-7ce02ed80334 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:24:52.674591 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 458/897] 10.222.0.33 () {56 vars in 1109 bytes} [Mon Jul 25 05:24:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 05:24:52.679786 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c34d62-c780-4209-8a2c-7ce02ed80334 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:24:52.714859 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:24:52.745955 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:24:52.764582 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3935534e-a958-44cf-90a3-8a1ff4f12e38 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:24:52.794423 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c34d62-c780-4209-8a2c-7ce02ed80334 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:24:52.809255 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de26d52e-1743-4a0a-ad35-29179ed6f12c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:24:52.810187 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de26d52e-1743-4a0a-ad35-29179ed6f12c None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:24:52.810669 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de26d52e-1743-4a0a-ad35-29179ed6f12c None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:24:52.811191 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de26d52e-1743-4a0a-ad35-29179ed6f12c None admin] PATH_INFO: `/v3/endpoints` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:24:52.812287 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de26d52e-1743-4a0a-ad35-29179ed6f12c None admin] RBAC: Adding query filter params (interface=public, service_id=97f286989e5a4a9c9a55421ff313327d, region_id=RegionOne) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:24:52.812812 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de26d52e-1743-4a0a-ad35-29179ed6f12c None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:24:52.814741 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de26d52e-1743-4a0a-ad35-29179ed6f12c None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:24:52.825737 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 440/898] 10.222.0.33 () {56 vars in 1251 bytes} [Mon Jul 25 05:24:52 2022] GET /identity/v3/endpoints?service_id=97f286989e5a4a9c9a55421ff313327d&interface=public®ion_id=RegionOne => generated 194 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:24:52.826895 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de26d52e-1743-4a0a-ad35-29179ed6f12c None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:24:52.834483 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebba0379-979a-4121-815a-158921bb90f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:24:52.835285 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebba0379-979a-4121-815a-158921bb90f3 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:24:52.835573 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebba0379-979a-4121-815a-158921bb90f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:24:52.835850 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebba0379-979a-4121-815a-158921bb90f3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:24:52.844409 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebba0379-979a-4121-815a-158921bb90f3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:24:52.845945 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebba0379-979a-4121-815a-158921bb90f3 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:24:52.861808 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application [None req-ebba0379-979a-4121-815a-158921bb90f3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 05:24:52.861808 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:24:52.861808 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:24:52.861808 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:24:52.861808 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:24:52.861808 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:24:52.861808 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:24:52.861808 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:24:52.861808 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:24:52.861808 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:24:52.861808 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:24:52.861808 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:24:52.861808 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 25 05:24:52.861808 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 25 05:24:52.861808 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 25 05:24:52.862714 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 25 05:24:52.862714 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:24:52.862714 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:24:52.862714 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:24:52.862714 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:24:52.862714 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:24:52.862714 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:24:52.862714 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:24:52.862714 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application with Lock( Jul 25 05:24:52.862714 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:24:52.862714 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:24:52.862714 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:24:52.862714 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:24:52.862714 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:24:52.862714 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:24:52.862714 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:24:52.862714 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:24:52.863606 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 25 05:24:52.863606 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 25 05:24:52.863606 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 25 05:24:52.863606 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 25 05:24:52.863606 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 25 05:24:52.863606 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 25 05:24:52.863606 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 05:24:52.863606 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Jul 25 05:24:52.863606 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 459/899] 10.222.0.33 () {56 vars in 1111 bytes} [Mon Jul 25 05:24:52 2022] GET /identity/v3/projects/service => generated 88 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:24:52.868911 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebba0379-979a-4121-815a-158921bb90f3 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:24:52.937190 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de26d52e-1743-4a0a-ad35-29179ed6f12c None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:24:52.959191 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:24:52.974912 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a8e040-db8d-4f59-8695-3b837accf974 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:24:52.975714 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a8e040-db8d-4f59-8695-3b837accf974 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:24:52.976098 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a8e040-db8d-4f59-8695-3b837accf974 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:24:52.976469 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a8e040-db8d-4f59-8695-3b837accf974 None admin] PATH_INFO: `/v3/services` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:24:52.977393 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54a8e040-db8d-4f59-8695-3b837accf974 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:24:52.978943 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54a8e040-db8d-4f59-8695-3b837accf974 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:24:52.984463 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebba0379-979a-4121-815a-158921bb90f3 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:24:52.993247 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 441/900] 10.222.0.33 () {56 vars in 1087 bytes} [Mon Jul 25 05:24:52 2022] GET /identity/v3/services => generated 1268 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 05:24:53.004602 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:24:53.025939 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8565dcfa-1dc7-468a-9c33-d73446c7b5a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:24:53.027689 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8565dcfa-1dc7-468a-9c33-d73446c7b5a0 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:24:53.028144 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8565dcfa-1dc7-468a-9c33-d73446c7b5a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:24:53.028573 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8565dcfa-1dc7-468a-9c33-d73446c7b5a0 None admin] PATH_INFO: `/v3/projects` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:24:53.029604 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8565dcfa-1dc7-468a-9c33-d73446c7b5a0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:24:53.030070 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8565dcfa-1dc7-468a-9c33-d73446c7b5a0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:24:53.031688 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8565dcfa-1dc7-468a-9c33-d73446c7b5a0 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:24:53.046229 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 460/901] 10.222.0.33 () {56 vars in 1148 bytes} [Mon Jul 25 05:24:52 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:24:53.052725 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a8e040-db8d-4f59-8695-3b837accf974 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:24:53.168597 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a8e040-db8d-4f59-8695-3b837accf974 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:24:53.204055 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caec186f-2195-48e8-a36b-ce89fce1d73c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:24:53.204710 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caec186f-2195-48e8-a36b-ce89fce1d73c None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:24:53.205018 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caec186f-2195-48e8-a36b-ce89fce1d73c None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:24:53.205289 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caec186f-2195-48e8-a36b-ce89fce1d73c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:24:53.206170 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caec186f-2195-48e8-a36b-ce89fce1d73c None admin] RBAC: Adding query filter params (role.id=e85cbdc7a77a4fc29d71c8a3ae0f56f4, scope.project.id=92afba266cce4d1bad4134c351e74be3, user.id=3352531be1b54f76b864383ccc2e5ec2) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:24:53.206433 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caec186f-2195-48e8-a36b-ce89fce1d73c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:24:53.207854 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caec186f-2195-48e8-a36b-ce89fce1d73c None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:24:53.214570 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 442/902] 10.222.0.33 () {56 vars in 1374 bytes} [Mon Jul 25 05:24:53 2022] GET /identity/v3/role_assignments?user.id=3352531be1b54f76b864383ccc2e5ec2&scope.project.id=92afba266cce4d1bad4134c351e74be3&role.id=e85cbdc7a77a4fc29d71c8a3ae0f56f4 => generated 259 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:24:53.513834 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd46192-7227-4cf3-9519-c2f05e2ed07e None None] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:24:53.514029 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd46192-7227-4cf3-9519-c2f05e2ed07e None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:24:53.514320 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd46192-7227-4cf3-9519-c2f05e2ed07e None None] PATH_INFO: `/` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:24:53.515978 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 461/903] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jul 25 05:24:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:24:53.521577 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c606a1-d26c-4912-a295-024f0282bd4e None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:24:53.521876 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c606a1-d26c-4912-a295-024f0282bd4e None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:24:53.522160 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c606a1-d26c-4912-a295-024f0282bd4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:24:53.539764 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-04c606a1-d26c-4912-a295-024f0282bd4e None None] MFA Rules not processed for user `1eae540242d94c10b0ad33960a855f08`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:24:53.541007 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-04c606a1-d26c-4912-a295-024f0282bd4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:24:53.589972 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:24:53.619385 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 443/904] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jul 25 05:24:53 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 25 05:24:53.638078 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd46192-7227-4cf3-9519-c2f05e2ed07e None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:24:53.746357 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd46192-7227-4cf3-9519-c2f05e2ed07e None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:24:53.783005 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-936e1d77-305a-4200-a2da-bc1217272bfb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:24:53.783854 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936e1d77-305a-4200-a2da-bc1217272bfb None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:24:53.784144 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936e1d77-305a-4200-a2da-bc1217272bfb None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:24:53.784425 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936e1d77-305a-4200-a2da-bc1217272bfb None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:24:53.785120 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-936e1d77-305a-4200-a2da-bc1217272bfb None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:24:53.786631 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-936e1d77-305a-4200-a2da-bc1217272bfb None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:24:53.800297 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application [None req-936e1d77-305a-4200-a2da-bc1217272bfb None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 25 05:24:53.800297 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:24:53.800297 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 25 05:24:53.800297 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 25 05:24:53.800297 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 25 05:24:53.800297 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 25 05:24:53.800297 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 25 05:24:53.800297 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 25 05:24:53.800297 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 25 05:24:53.800297 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Jul 25 05:24:53.800297 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 25 05:24:53.800297 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Jul 25 05:24:53.800297 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:24:53.800297 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:24:53.800297 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:24:53.800297 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:24:53.801319 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:24:53.801319 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:24:53.801319 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:24:53.801319 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:24:53.801319 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:24:53.801319 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:24:53.801319 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:24:53.801319 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 25 05:24:53.801319 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 25 05:24:53.801319 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 25 05:24:53.801319 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 25 05:24:53.801319 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:24:53.801319 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:24:53.801319 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:24:53.801319 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:24:53.801319 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:24:53.802150 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:24:53.802150 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:24:53.802150 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application with Lock( Jul 25 05:24:53.802150 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:24:53.802150 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:24:53.802150 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:24:53.802150 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:24:53.802150 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:24:53.802150 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:24:53.802150 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:24:53.802150 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:24:53.802150 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 25 05:24:53.802150 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 25 05:24:53.802150 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 25 05:24:53.802150 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Jul 25 05:24:53.802150 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 462/905] 10.222.0.33 () {56 vars in 1132 bytes} [Mon Jul 25 05:24:53 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:24:53.807819 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c606a1-d26c-4912-a295-024f0282bd4e None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:24:53.914622 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c606a1-d26c-4912-a295-024f0282bd4e None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:24:53.952044 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-805a2682-16e9-434e-9062-1d36013f9cac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:24:53.953130 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805a2682-16e9-434e-9062-1d36013f9cac None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:24:53.953636 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805a2682-16e9-434e-9062-1d36013f9cac None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:24:53.954111 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805a2682-16e9-434e-9062-1d36013f9cac None admin] PATH_INFO: `/v3/services` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:24:53.955015 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-805a2682-16e9-434e-9062-1d36013f9cac None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:24:53.955539 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-805a2682-16e9-434e-9062-1d36013f9cac None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:24:53.957128 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-805a2682-16e9-434e-9062-1d36013f9cac None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:24:53.962598 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 444/906] 10.222.0.33 () {56 vars in 1126 bytes} [Mon Jul 25 05:24:53 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:24:53.968891 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-936e1d77-305a-4200-a2da-bc1217272bfb None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:24:54.013449 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:24:54.074397 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-936e1d77-305a-4200-a2da-bc1217272bfb None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:24:54.077458 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a271a2d8-1937-48e9-ba84-cecc422083e9 None None] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:24:54.077751 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a271a2d8-1937-48e9-ba84-cecc422083e9 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:24:54.078032 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a271a2d8-1937-48e9-ba84-cecc422083e9 None None] PATH_INFO: `/` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:24:54.079655 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 445/907] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jul 25 05:24:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 25 05:24:54.084576 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca50e5f8-ca97-42bb-912d-9c0601b70822 None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:24:54.084892 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca50e5f8-ca97-42bb-912d-9c0601b70822 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:24:54.085174 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca50e5f8-ca97-42bb-912d-9c0601b70822 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:24:54.104640 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-ca50e5f8-ca97-42bb-912d-9c0601b70822 None None] MFA Rules not processed for user `1eae540242d94c10b0ad33960a855f08`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:24:54.106051 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-ca50e5f8-ca97-42bb-912d-9c0601b70822 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:24:54.118153 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c8a99d-db61-468d-8b69-24face2934ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:24:54.119365 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c8a99d-db61-468d-8b69-24face2934ef None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:24:54.119870 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c8a99d-db61-468d-8b69-24face2934ef None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:24:54.120619 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c8a99d-db61-468d-8b69-24face2934ef None admin] PATH_INFO: `/v3/services` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:24:54.121990 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94c8a99d-db61-468d-8b69-24face2934ef None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:24:54.122718 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94c8a99d-db61-468d-8b69-24face2934ef None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:24:54.124909 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94c8a99d-db61-468d-8b69-24face2934ef None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:24:54.131691 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 463/908] 10.222.0.33 () {56 vars in 1126 bytes} [Mon Jul 25 05:24:53 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:24:54.137502 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c8a99d-db61-468d-8b69-24face2934ef None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:24:54.146601 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:24:54.184001 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 446/909] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jul 25 05:24:54 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 25 05:24:54.206915 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca50e5f8-ca97-42bb-912d-9c0601b70822 None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:24:54.258802 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c8a99d-db61-468d-8b69-24face2934ef None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:24:54.262124 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:24:54.307053 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a50e87f-293b-48ec-99c5-0e7c327224d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:24:54.308284 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a50e87f-293b-48ec-99c5-0e7c327224d6 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:24:54.309063 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a50e87f-293b-48ec-99c5-0e7c327224d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:24:54.309352 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a50e87f-293b-48ec-99c5-0e7c327224d6 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:24:54.310431 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a50e87f-293b-48ec-99c5-0e7c327224d6 None admin] RBAC: Adding query filter params (interface=public, service_id=5b38596d9fa7443db0b38695fef45852, region_id=RegionOne) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:24:54.310751 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a50e87f-293b-48ec-99c5-0e7c327224d6 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:24:54.312769 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a50e87f-293b-48ec-99c5-0e7c327224d6 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:24:54.318764 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca50e5f8-ca97-42bb-912d-9c0601b70822 None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:24:54.319832 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 464/910] 10.222.0.33 () {56 vars in 1251 bytes} [Mon Jul 25 05:24:54 2022] GET /identity/v3/endpoints?service_id=5b38596d9fa7443db0b38695fef45852&interface=public®ion_id=RegionOne => generated 194 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:24:54.332881 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a50e87f-293b-48ec-99c5-0e7c327224d6 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:24:54.365728 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46327d05-6777-416a-981b-00bbe653cf48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:24:54.367063 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46327d05-6777-416a-981b-00bbe653cf48 None admin] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:24:54.367351 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46327d05-6777-416a-981b-00bbe653cf48 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:24:54.367630 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46327d05-6777-416a-981b-00bbe653cf48 None admin] PATH_INFO: `/v3/services` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:24:54.368831 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46327d05-6777-416a-981b-00bbe653cf48 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:24:54.370828 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46327d05-6777-416a-981b-00bbe653cf48 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:24:54.392955 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 447/911] 10.222.0.33 () {60 vars in 1141 bytes} [Mon Jul 25 05:24:54 2022] POST /identity/v3/services => generated 241 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 25 05:24:54.441471 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a50e87f-293b-48ec-99c5-0e7c327224d6 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:24:54.485173 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c31bd4d-dfd9-4f6a-835d-e601d47a795d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:24:54.485853 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c31bd4d-dfd9-4f6a-835d-e601d47a795d None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:24:54.486135 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c31bd4d-dfd9-4f6a-835d-e601d47a795d None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:24:54.486409 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c31bd4d-dfd9-4f6a-835d-e601d47a795d None admin] PATH_INFO: `/v3/services` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:24:54.487098 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c31bd4d-dfd9-4f6a-835d-e601d47a795d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:24:54.488489 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c31bd4d-dfd9-4f6a-835d-e601d47a795d None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:24:54.496184 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 465/912] 10.222.0.33 () {56 vars in 1087 bytes} [Mon Jul 25 05:24:54 2022] GET /identity/v3/services => generated 1497 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 25 05:24:55.201942 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dafe8fad-7b6e-43ca-8824-606009a093fa None None] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:24:55.202945 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dafe8fad-7b6e-43ca-8824-606009a093fa None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:24:55.203468 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dafe8fad-7b6e-43ca-8824-606009a093fa None None] PATH_INFO: `/` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:24:55.205380 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 448/913] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jul 25 05:24:55 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:24:55.210989 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5721163-557a-4f05-9ceb-5c5ddebacdb4 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:24:55.211842 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5721163-557a-4f05-9ceb-5c5ddebacdb4 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:24:55.212344 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5721163-557a-4f05-9ceb-5c5ddebacdb4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:24:55.230084 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-e5721163-557a-4f05-9ceb-5c5ddebacdb4 None None] MFA Rules not processed for user `1eae540242d94c10b0ad33960a855f08`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:24:55.231613 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-e5721163-557a-4f05-9ceb-5c5ddebacdb4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:24:55.256035 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:24:55.304468 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 466/914] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jul 25 05:24:55 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 25 05:24:55.320576 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b13d16-6de6-42d1-a98e-21ede9cf6286 None None] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:24:55.321144 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b13d16-6de6-42d1-a98e-21ede9cf6286 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:24:55.321663 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b13d16-6de6-42d1-a98e-21ede9cf6286 None None] PATH_INFO: `/` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:24:55.323338 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 449/915] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jul 25 05:24:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:24:55.328662 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67fcf40b-d3b4-4a56-802e-a999205feb23 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:24:55.329155 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67fcf40b-d3b4-4a56-802e-a999205feb23 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:24:55.329661 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67fcf40b-d3b4-4a56-802e-a999205feb23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:24:55.334368 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b13d16-6de6-42d1-a98e-21ede9cf6286 None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:24:55.348233 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-67fcf40b-d3b4-4a56-802e-a999205feb23 None None] MFA Rules not processed for user `1eae540242d94c10b0ad33960a855f08`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:24:55.349461 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-67fcf40b-d3b4-4a56-802e-a999205feb23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:24:55.374223 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:24:55.421270 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 467/916] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jul 25 05:24:55 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 25 05:24:55.449411 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67fcf40b-d3b4-4a56-802e-a999205feb23 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:24:55.466568 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b13d16-6de6-42d1-a98e-21ede9cf6286 None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:24:55.515588 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c489cd1d-48b2-4d5e-a23d-fe0c23c8b4eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:24:55.516652 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c489cd1d-48b2-4d5e-a23d-fe0c23c8b4eb None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:24:55.517378 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c489cd1d-48b2-4d5e-a23d-fe0c23c8b4eb None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:24:55.517668 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c489cd1d-48b2-4d5e-a23d-fe0c23c8b4eb None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:24:55.518633 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c489cd1d-48b2-4d5e-a23d-fe0c23c8b4eb None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:24:55.520691 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c489cd1d-48b2-4d5e-a23d-fe0c23c8b4eb None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:24:55.537244 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application [None req-c489cd1d-48b2-4d5e-a23d-fe0c23c8b4eb None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 25 05:24:55.537244 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:24:55.537244 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 25 05:24:55.537244 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 25 05:24:55.537244 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 25 05:24:55.537244 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 25 05:24:55.537244 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 25 05:24:55.537244 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 25 05:24:55.537244 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 25 05:24:55.537244 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Jul 25 05:24:55.537244 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 25 05:24:55.537244 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Jul 25 05:24:55.537244 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:24:55.537244 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:24:55.537244 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:24:55.537244 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:24:55.537244 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:24:55.538302 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:24:55.538302 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:24:55.538302 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:24:55.538302 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:24:55.538302 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:24:55.538302 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:24:55.538302 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 25 05:24:55.538302 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 25 05:24:55.538302 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 25 05:24:55.538302 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 25 05:24:55.538302 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:24:55.538302 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:24:55.538302 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:24:55.538302 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:24:55.538302 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:24:55.538302 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:24:55.539165 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:24:55.539165 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application with Lock( Jul 25 05:24:55.539165 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:24:55.539165 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:24:55.539165 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:24:55.539165 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:24:55.539165 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:24:55.539165 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:24:55.539165 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:24:55.539165 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:24:55.539165 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 25 05:24:55.539165 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 25 05:24:55.539165 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 25 05:24:55.539165 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Jul 25 05:24:55.544326 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 450/917] 10.222.0.33 () {56 vars in 1114 bytes} [Mon Jul 25 05:24:55 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:24:55.545812 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c489cd1d-48b2-4d5e-a23d-fe0c23c8b4eb None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:24:55.559109 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:24:55.578127 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67fcf40b-d3b4-4a56-802e-a999205feb23 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:24:55.628461 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4dd76f0-8118-4851-9024-1878ce43b739 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:24:55.629615 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4dd76f0-8118-4851-9024-1878ce43b739 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:24:55.629908 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4dd76f0-8118-4851-9024-1878ce43b739 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:24:55.630194 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4dd76f0-8118-4851-9024-1878ce43b739 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:24:55.645254 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4dd76f0-8118-4851-9024-1878ce43b739 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 05:24:55.657063 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application [None req-c4dd76f0-8118-4851-9024-1878ce43b739 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 25 05:24:55.657063 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:24:55.657063 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:24:55.657063 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:24:55.657063 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:24:55.657063 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:24:55.657063 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:24:55.657063 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:24:55.657063 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:24:55.657063 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:24:55.657063 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:24:55.657063 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:24:55.657063 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 05:24:55.657063 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 05:24:55.657063 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 05:24:55.657063 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 05:24:55.658283 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 05:24:55.658283 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 05:24:55.658283 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 05:24:55.658283 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 05:24:55.658283 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:24:55.658283 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:24:55.658283 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:24:55.658283 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:24:55.658283 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:24:55.658283 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:24:55.658283 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:24:55.658283 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application with Lock( Jul 25 05:24:55.658283 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:24:55.658283 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:24:55.658283 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:24:55.658283 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:24:55.659191 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:24:55.659191 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:24:55.659191 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:24:55.659191 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:24:55.659191 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 05:24:55.659191 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 05:24:55.659191 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 05:24:55.659191 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 05:24:55.659191 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 05:24:55.659191 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 05:24:55.659191 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 25 05:24:55.659191 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Jul 25 05:24:55.659191 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 468/918] 10.222.0.33 () {56 vars in 1120 bytes} [Mon Jul 25 05:24:55 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:24:55.664475 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4dd76f0-8118-4851-9024-1878ce43b739 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:24:55.667373 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:24:55.674776 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c489cd1d-48b2-4d5e-a23d-fe0c23c8b4eb None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:24:55.710744 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-924a34d0-dfa5-46eb-bff9-232e6c3e9648 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:24:55.711587 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924a34d0-dfa5-46eb-bff9-232e6c3e9648 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:24:55.711967 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924a34d0-dfa5-46eb-bff9-232e6c3e9648 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:24:55.712350 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924a34d0-dfa5-46eb-bff9-232e6c3e9648 None admin] PATH_INFO: `/v3/services` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:24:55.713194 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-924a34d0-dfa5-46eb-bff9-232e6c3e9648 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:24:55.713601 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-924a34d0-dfa5-46eb-bff9-232e6c3e9648 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:24:55.715084 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-924a34d0-dfa5-46eb-bff9-232e6c3e9648 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:24:55.720717 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 451/919] 10.222.0.33 () {56 vars in 1114 bytes} [Mon Jul 25 05:24:55 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:24:55.727401 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-924a34d0-dfa5-46eb-bff9-232e6c3e9648 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:24:55.789661 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4dd76f0-8118-4851-9024-1878ce43b739 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:24:55.830683 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df4b8f01-b4d8-4b83-b23b-9f2fb36f8ee2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:24:55.831560 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4b8f01-b4d8-4b83-b23b-9f2fb36f8ee2 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:24:55.831989 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4b8f01-b4d8-4b83-b23b-9f2fb36f8ee2 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:24:55.832474 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4b8f01-b4d8-4b83-b23b-9f2fb36f8ee2 None admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:24:55.833478 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df4b8f01-b4d8-4b83-b23b-9f2fb36f8ee2 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:24:55.834000 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df4b8f01-b4d8-4b83-b23b-9f2fb36f8ee2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:24:55.835551 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df4b8f01-b4d8-4b83-b23b-9f2fb36f8ee2 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:24:55.849085 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 469/920] 10.222.0.33 () {56 vars in 1115 bytes} [Mon Jul 25 05:24:55 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 05:24:55.850507 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df4b8f01-b4d8-4b83-b23b-9f2fb36f8ee2 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:24:55.865484 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:24:55.870676 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-924a34d0-dfa5-46eb-bff9-232e6c3e9648 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:24:55.919494 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c3afff-f989-4a96-94e2-f93591b702c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:24:55.920175 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c3afff-f989-4a96-94e2-f93591b702c7 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:24:55.920464 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c3afff-f989-4a96-94e2-f93591b702c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:24:55.920746 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c3afff-f989-4a96-94e2-f93591b702c7 None admin] PATH_INFO: `/v3/services` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:24:55.921973 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43c3afff-f989-4a96-94e2-f93591b702c7 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:24:55.922634 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43c3afff-f989-4a96-94e2-f93591b702c7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:24:55.924346 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43c3afff-f989-4a96-94e2-f93591b702c7 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:24:55.933067 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 452/921] 10.222.0.33 () {56 vars in 1114 bytes} [Mon Jul 25 05:24:55 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:24:55.939421 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c3afff-f989-4a96-94e2-f93591b702c7 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:24:55.974365 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df4b8f01-b4d8-4b83-b23b-9f2fb36f8ee2 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:24:55.974878 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:24:56.011643 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-569a3c49-7788-4303-bebb-b621c5c483fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:24:56.012837 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569a3c49-7788-4303-bebb-b621c5c483fe None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:24:56.013365 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569a3c49-7788-4303-bebb-b621c5c483fe None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:24:56.013859 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569a3c49-7788-4303-bebb-b621c5c483fe None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:24:56.027420 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-569a3c49-7788-4303-bebb-b621c5c483fe None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:24:56.028897 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-569a3c49-7788-4303-bebb-b621c5c483fe None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:24:56.047247 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application [None req-569a3c49-7788-4303-bebb-b621c5c483fe None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 05:24:56.047247 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:24:56.047247 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:24:56.047247 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:24:56.047247 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:24:56.047247 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:24:56.047247 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:24:56.047247 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:24:56.047247 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:24:56.047247 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:24:56.047247 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:24:56.047247 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:24:56.047247 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 05:24:56.047247 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 05:24:56.047247 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 05:24:56.048192 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 05:24:56.048192 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:24:56.048192 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:24:56.048192 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:24:56.048192 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:24:56.048192 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:24:56.048192 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:24:56.048192 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:24:56.048192 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application with Lock( Jul 25 05:24:56.048192 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:24:56.048192 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:24:56.048192 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:24:56.048192 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:24:56.048192 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:24:56.048192 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:24:56.048192 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:24:56.048192 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:24:56.049233 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 05:24:56.049233 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 05:24:56.049233 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 05:24:56.049233 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Jul 25 05:24:56.056008 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 470/922] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Jul 25 05:24:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 25 05:24:56.057155 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-569a3c49-7788-4303-bebb-b621c5c483fe None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:24:56.076153 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c3afff-f989-4a96-94e2-f93591b702c7 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:24:56.124389 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:24:56.127400 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f704bee2-2800-43dd-bc24-d4fee5891075 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:24:56.128126 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f704bee2-2800-43dd-bc24-d4fee5891075 None admin] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:24:56.128423 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f704bee2-2800-43dd-bc24-d4fee5891075 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:24:56.128706 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f704bee2-2800-43dd-bc24-d4fee5891075 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:24:56.129533 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f704bee2-2800-43dd-bc24-d4fee5891075 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:24:56.131248 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f704bee2-2800-43dd-bc24-d4fee5891075 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:24:56.164864 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 453/923] 10.222.0.33 () {60 vars in 1144 bytes} [Mon Jul 25 05:24:55 2022] POST /identity/v3/endpoints => generated 346 bytes in 224 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 05:24:56.175390 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-569a3c49-7788-4303-bebb-b621c5c483fe None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:24:56.224517 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9229ea1-a191-4233-a4dc-8afffcd4c1d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:24:56.226059 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9229ea1-a191-4233-a4dc-8afffcd4c1d6 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:24:56.226578 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9229ea1-a191-4233-a4dc-8afffcd4c1d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:24:56.227051 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9229ea1-a191-4233-a4dc-8afffcd4c1d6 None admin] PATH_INFO: `/v3/domains` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:24:56.228063 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9229ea1-a191-4233-a4dc-8afffcd4c1d6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:24:56.228585 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9229ea1-a191-4233-a4dc-8afffcd4c1d6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:24:56.230324 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9229ea1-a191-4233-a4dc-8afffcd4c1d6 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:24:56.244857 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 471/924] 10.222.0.33 () {56 vars in 1109 bytes} [Mon Jul 25 05:24:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:24:56.251544 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f704bee2-2800-43dd-bc24-d4fee5891075 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:24:56.344277 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:24:56.369143 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f704bee2-2800-43dd-bc24-d4fee5891075 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:24:56.405381 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b21aaa-36e8-420a-b7af-6aac55952766 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:24:56.406195 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b21aaa-36e8-420a-b7af-6aac55952766 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:24:56.406477 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b21aaa-36e8-420a-b7af-6aac55952766 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:24:56.406751 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b21aaa-36e8-420a-b7af-6aac55952766 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:24:56.414729 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21b21aaa-36e8-420a-b7af-6aac55952766 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:24:56.416158 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21b21aaa-36e8-420a-b7af-6aac55952766 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:24:56.431899 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application [None req-21b21aaa-36e8-420a-b7af-6aac55952766 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 25 05:24:56.431899 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:24:56.431899 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:24:56.431899 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:24:56.431899 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:24:56.431899 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:24:56.431899 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:24:56.431899 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:24:56.431899 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:24:56.431899 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:24:56.431899 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:24:56.431899 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:24:56.431899 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 25 05:24:56.431899 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 25 05:24:56.431899 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 25 05:24:56.431899 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 25 05:24:56.433050 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:24:56.433050 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:24:56.433050 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 25 05:24:56.433050 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 05:24:56.433050 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 25 05:24:56.433050 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 05:24:56.433050 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:24:56.433050 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:24:56.433050 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:24:56.433050 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:24:56.433050 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:24:56.433050 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application with Lock( Jul 25 05:24:56.433050 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:24:56.433050 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:24:56.433050 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:24:56.433050 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:24:56.433050 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:24:56.433050 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:24:56.434810 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:24:56.434810 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:24:56.434810 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 25 05:24:56.434810 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 25 05:24:56.434810 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 25 05:24:56.434810 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 25 05:24:56.434810 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 25 05:24:56.434810 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 25 05:24:56.434810 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 25 05:24:56.434810 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Jul 25 05:24:56.434810 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 454/925] 10.222.0.33 () {56 vars in 1117 bytes} [Mon Jul 25 05:24:56 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:24:56.439743 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9229ea1-a191-4233-a4dc-8afffcd4c1d6 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:24:56.523317 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feedb8f8-f9db-41fc-b468-0060d716cabe None None] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:24:56.523622 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feedb8f8-f9db-41fc-b468-0060d716cabe None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:24:56.523851 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feedb8f8-f9db-41fc-b468-0060d716cabe None None] PATH_INFO: `/` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:24:56.525662 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 455/926] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jul 25 05:24:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:24:56.531703 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ca2bbf-5006-4f85-8e94-7464077119fa None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:24:56.532063 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ca2bbf-5006-4f85-8e94-7464077119fa None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:24:56.532353 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ca2bbf-5006-4f85-8e94-7464077119fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:24:56.547740 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9229ea1-a191-4233-a4dc-8afffcd4c1d6 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:24:56.551555 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-52ca2bbf-5006-4f85-8e94-7464077119fa None None] MFA Rules not processed for user `1eae540242d94c10b0ad33960a855f08`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:24:56.552889 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-52ca2bbf-5006-4f85-8e94-7464077119fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:24:56.567436 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:24:56.587137 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-751f8aa7-3d11-48d6-8bee-38344ba50b8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:24:56.588569 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751f8aa7-3d11-48d6-8bee-38344ba50b8f None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:24:56.589692 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751f8aa7-3d11-48d6-8bee-38344ba50b8f None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:24:56.590117 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751f8aa7-3d11-48d6-8bee-38344ba50b8f None admin] PATH_INFO: `/v3/users` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:24:56.591262 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-751f8aa7-3d11-48d6-8bee-38344ba50b8f None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:24:56.592018 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-751f8aa7-3d11-48d6-8bee-38344ba50b8f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:24:56.593932 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-751f8aa7-3d11-48d6-8bee-38344ba50b8f None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:24:56.616062 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 472/927] 10.222.0.33 () {56 vars in 1149 bytes} [Mon Jul 25 05:24:56 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:24:56.617648 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da83b26a-a97c-400b-a4d3-464151aad72c None None] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:24:56.618083 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da83b26a-a97c-400b-a4d3-464151aad72c None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:24:56.618941 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da83b26a-a97c-400b-a4d3-464151aad72c None None] PATH_INFO: `/` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:24:56.621050 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 473/928] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jul 25 05:24:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:24:56.622122 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da83b26a-a97c-400b-a4d3-464151aad72c None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:24:56.634159 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 456/929] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jul 25 05:24:56 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 25 05:24:56.638325 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6441d9-79a2-4650-9971-3ee58a9d2a02 None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:24:56.638626 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6441d9-79a2-4650-9971-3ee58a9d2a02 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:24:56.638912 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6441d9-79a2-4650-9971-3ee58a9d2a02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:24:56.657151 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-7b6441d9-79a2-4650-9971-3ee58a9d2a02 None None] MFA Rules not processed for user `1eae540242d94c10b0ad33960a855f08`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:24:56.658361 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-7b6441d9-79a2-4650-9971-3ee58a9d2a02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:24:56.725905 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:24:56.731994 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 457/930] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jul 25 05:24:56 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 25 05:24:56.732793 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b6441d9-79a2-4650-9971-3ee58a9d2a02 None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:24:56.742191 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da83b26a-a97c-400b-a4d3-464151aad72c None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:24:56.782908 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-281a3661-3eef-4240-a4e3-c4c321c4e33b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:24:56.783626 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281a3661-3eef-4240-a4e3-c4c321c4e33b None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:24:56.783913 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281a3661-3eef-4240-a4e3-c4c321c4e33b None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:24:56.784195 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281a3661-3eef-4240-a4e3-c4c321c4e33b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:24:56.795395 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-281a3661-3eef-4240-a4e3-c4c321c4e33b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:24:56.796871 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-281a3661-3eef-4240-a4e3-c4c321c4e33b None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:24:56.820499 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application [None req-281a3661-3eef-4240-a4e3-c4c321c4e33b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 05:24:56.820499 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:24:56.820499 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:24:56.820499 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:24:56.820499 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:24:56.820499 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:24:56.820499 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:24:56.820499 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:24:56.820499 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:24:56.820499 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:24:56.820499 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:24:56.820499 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:24:56.820499 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 05:24:56.820499 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 05:24:56.820499 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 05:24:56.821593 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 05:24:56.821593 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:24:56.821593 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:24:56.821593 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:24:56.821593 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:24:56.821593 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:24:56.821593 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:24:56.821593 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:24:56.821593 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application with Lock( Jul 25 05:24:56.821593 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:24:56.821593 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:24:56.821593 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:24:56.821593 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:24:56.821593 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:24:56.821593 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:24:56.821593 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:24:56.821593 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:24:56.822485 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 05:24:56.822485 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 05:24:56.822485 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 05:24:56.822485 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Jul 25 05:24:56.822485 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 474/931] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Jul 25 05:24:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:24:56.822485 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-281a3661-3eef-4240-a4e3-c4c321c4e33b None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:24:56.849043 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b6441d9-79a2-4650-9971-3ee58a9d2a02 None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:24:56.861498 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:24:56.884335 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798a13b7-a5b2-463f-88f9-8a4e5bc1fb82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:24:56.885326 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798a13b7-a5b2-463f-88f9-8a4e5bc1fb82 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:24:56.885812 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798a13b7-a5b2-463f-88f9-8a4e5bc1fb82 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:24:56.886105 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798a13b7-a5b2-463f-88f9-8a4e5bc1fb82 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:24:56.886785 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-798a13b7-a5b2-463f-88f9-8a4e5bc1fb82 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:24:56.888197 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-798a13b7-a5b2-463f-88f9-8a4e5bc1fb82 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:24:56.901059 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application [None req-798a13b7-a5b2-463f-88f9-8a4e5bc1fb82 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 25 05:24:56.901059 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:24:56.901059 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 25 05:24:56.901059 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 25 05:24:56.901059 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 25 05:24:56.901059 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 25 05:24:56.901059 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 25 05:24:56.901059 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 25 05:24:56.901059 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 25 05:24:56.901059 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Jul 25 05:24:56.901059 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 25 05:24:56.901059 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Jul 25 05:24:56.901059 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:24:56.901059 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:24:56.901059 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:24:56.901059 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:24:56.901059 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:24:56.902928 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:24:56.902928 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:24:56.902928 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:24:56.902928 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:24:56.902928 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:24:56.902928 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:24:56.902928 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 25 05:24:56.902928 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 25 05:24:56.902928 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 25 05:24:56.902928 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 25 05:24:56.902928 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:24:56.902928 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:24:56.902928 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:24:56.902928 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:24:56.902928 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:24:56.902928 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:24:56.903785 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:24:56.903785 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application with Lock( Jul 25 05:24:56.903785 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:24:56.903785 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:24:56.903785 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:24:56.903785 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:24:56.903785 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:24:56.903785 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:24:56.903785 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:24:56.903785 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:24:56.903785 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 25 05:24:56.903785 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 25 05:24:56.903785 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 25 05:24:56.903785 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Jul 25 05:24:56.907913 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 458/932] 10.222.0.33 () {56 vars in 1126 bytes} [Mon Jul 25 05:24:56 2022] GET /identity/v3/services/object-store => generated 93 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:24:56.908793 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798a13b7-a5b2-463f-88f9-8a4e5bc1fb82 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:24:56.945680 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-281a3661-3eef-4240-a4e3-c4c321c4e33b None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:24:56.976457 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:24:56.982808 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d758477-abdb-4101-9a6f-467e40fff7d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:24:56.983847 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d758477-abdb-4101-9a6f-467e40fff7d2 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:24:56.984136 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d758477-abdb-4101-9a6f-467e40fff7d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:24:56.984420 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d758477-abdb-4101-9a6f-467e40fff7d2 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:24:56.985105 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d758477-abdb-4101-9a6f-467e40fff7d2 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:24:56.986475 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d758477-abdb-4101-9a6f-467e40fff7d2 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:24:56.999515 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application [None req-5d758477-abdb-4101-9a6f-467e40fff7d2 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 25 05:24:56.999515 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:24:56.999515 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 25 05:24:56.999515 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 25 05:24:56.999515 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 25 05:24:56.999515 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 25 05:24:56.999515 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 25 05:24:56.999515 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 25 05:24:56.999515 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 25 05:24:56.999515 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Jul 25 05:24:56.999515 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 25 05:24:56.999515 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Jul 25 05:24:56.999515 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:24:56.999515 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:24:56.999515 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:24:56.999515 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:24:57.000547 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:24:57.000547 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:24:57.000547 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:24:57.000547 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:24:57.000547 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:24:57.000547 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:24:57.000547 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:24:57.000547 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 25 05:24:57.000547 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 25 05:24:57.000547 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 25 05:24:57.000547 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 25 05:24:57.000547 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:24:57.000547 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:24:57.000547 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:24:57.000547 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:24:57.000547 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:24:57.001438 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:24:57.001438 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:24:57.001438 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application with Lock( Jul 25 05:24:57.001438 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:24:57.001438 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:24:57.001438 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:24:57.001438 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:24:57.001438 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:24:57.001438 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:24:57.001438 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:24:57.001438 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:24:57.001438 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 25 05:24:57.001438 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 25 05:24:57.001438 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 25 05:24:57.001438 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Jul 25 05:24:57.001438 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 475/933] 10.222.0.33 () {56 vars in 1132 bytes} [Mon Jul 25 05:24:56 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:24:57.002313 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d758477-abdb-4101-9a6f-467e40fff7d2 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:24:57.023983 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798a13b7-a5b2-463f-88f9-8a4e5bc1fb82 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:24:57.058016 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d25d80c-9296-456d-b6c2-728e2ae63605 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:24:57.059444 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d25d80c-9296-456d-b6c2-728e2ae63605 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:24:57.059732 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d25d80c-9296-456d-b6c2-728e2ae63605 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:24:57.060016 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d25d80c-9296-456d-b6c2-728e2ae63605 None admin] PATH_INFO: `/v3/domains` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:24:57.060748 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d25d80c-9296-456d-b6c2-728e2ae63605 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:24:57.061093 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d25d80c-9296-456d-b6c2-728e2ae63605 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:24:57.062476 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d25d80c-9296-456d-b6c2-728e2ae63605 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:24:57.079123 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 459/934] 10.222.0.33 () {56 vars in 1109 bytes} [Mon Jul 25 05:24:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:24:57.079123 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d25d80c-9296-456d-b6c2-728e2ae63605 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:24:57.111037 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:24:57.121744 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d758477-abdb-4101-9a6f-467e40fff7d2 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:24:57.158825 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd2bccb-0a95-4149-8a40-24f843854b42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:24:57.159604 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd2bccb-0a95-4149-8a40-24f843854b42 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:24:57.159893 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd2bccb-0a95-4149-8a40-24f843854b42 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:24:57.160285 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd2bccb-0a95-4149-8a40-24f843854b42 None admin] PATH_INFO: `/v3/services` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:24:57.161044 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdd2bccb-0a95-4149-8a40-24f843854b42 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:24:57.161358 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdd2bccb-0a95-4149-8a40-24f843854b42 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:24:57.162842 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdd2bccb-0a95-4149-8a40-24f843854b42 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:24:57.173247 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 476/935] 10.222.0.33 () {56 vars in 1122 bytes} [Mon Jul 25 05:24:57 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:24:57.174091 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd2bccb-0a95-4149-8a40-24f843854b42 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:24:57.175264 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d25d80c-9296-456d-b6c2-728e2ae63605 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:24:57.202779 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:24:57.215186 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22089d07-3598-4cac-805f-d6025bbb1f5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:24:57.215854 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22089d07-3598-4cac-805f-d6025bbb1f5b None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:24:57.216141 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22089d07-3598-4cac-805f-d6025bbb1f5b None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:24:57.216420 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22089d07-3598-4cac-805f-d6025bbb1f5b None admin] PATH_INFO: `/v3/services` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:24:57.217280 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22089d07-3598-4cac-805f-d6025bbb1f5b None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:24:57.217586 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22089d07-3598-4cac-805f-d6025bbb1f5b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:24:57.218956 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22089d07-3598-4cac-805f-d6025bbb1f5b None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:24:57.224659 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 460/936] 10.222.0.33 () {56 vars in 1126 bytes} [Mon Jul 25 05:24:57 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:24:57.225619 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22089d07-3598-4cac-805f-d6025bbb1f5b None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:24:57.296045 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd2bccb-0a95-4149-8a40-24f843854b42 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:24:57.337776 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3124ab38-0322-4dcc-9138-72cc74394fad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:24:57.338501 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3124ab38-0322-4dcc-9138-72cc74394fad None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:24:57.339289 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3124ab38-0322-4dcc-9138-72cc74394fad None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:24:57.339584 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3124ab38-0322-4dcc-9138-72cc74394fad None admin] PATH_INFO: `/v3/projects/service` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:24:57.340296 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22089d07-3598-4cac-805f-d6025bbb1f5b None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:24:57.346774 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3124ab38-0322-4dcc-9138-72cc74394fad None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:24:57.348215 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3124ab38-0322-4dcc-9138-72cc74394fad None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:24:57.364192 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application [None req-3124ab38-0322-4dcc-9138-72cc74394fad None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 05:24:57.364192 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:24:57.364192 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:24:57.364192 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:24:57.364192 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:24:57.364192 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:24:57.364192 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:24:57.364192 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:24:57.364192 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:24:57.364192 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:24:57.364192 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:24:57.364192 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:24:57.364192 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 25 05:24:57.364192 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 25 05:24:57.364192 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 25 05:24:57.365247 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 25 05:24:57.365247 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:24:57.365247 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:24:57.365247 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:24:57.365247 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:24:57.365247 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:24:57.365247 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:24:57.365247 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:24:57.365247 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application with Lock( Jul 25 05:24:57.365247 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:24:57.365247 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:24:57.365247 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:24:57.365247 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:24:57.365247 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:24:57.365247 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:24:57.365247 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:24:57.365247 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:24:57.366250 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 25 05:24:57.366250 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 25 05:24:57.366250 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 25 05:24:57.366250 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 25 05:24:57.366250 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 25 05:24:57.366250 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 25 05:24:57.366250 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 05:24:57.366250 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Jul 25 05:24:57.371473 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 477/937] 10.222.0.33 () {56 vars in 1111 bytes} [Mon Jul 25 05:24:57 2022] GET /identity/v3/projects/service => generated 88 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:24:57.372200 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3124ab38-0322-4dcc-9138-72cc74394fad None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:24:57.380162 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d59b0f42-e305-4f46-b785-c64f05d9923a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:24:57.381295 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59b0f42-e305-4f46-b785-c64f05d9923a None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:24:57.381587 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59b0f42-e305-4f46-b785-c64f05d9923a None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:24:57.381863 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59b0f42-e305-4f46-b785-c64f05d9923a None admin] PATH_INFO: `/v3/services` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:24:57.382946 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d59b0f42-e305-4f46-b785-c64f05d9923a None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:24:57.383268 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d59b0f42-e305-4f46-b785-c64f05d9923a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:24:57.385037 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d59b0f42-e305-4f46-b785-c64f05d9923a None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:24:57.390641 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:24:57.395984 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 461/938] 10.222.0.33 () {56 vars in 1122 bytes} [Mon Jul 25 05:24:57 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:24:57.396997 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d59b0f42-e305-4f46-b785-c64f05d9923a None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:24:57.455507 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:24:57.493312 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3124ab38-0322-4dcc-9138-72cc74394fad None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:24:57.504579 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d59b0f42-e305-4f46-b785-c64f05d9923a None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:24:57.534334 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d912ef-951c-4eb8-9855-5a0155ff19e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:24:57.535008 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d912ef-951c-4eb8-9855-5a0155ff19e0 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:24:57.535298 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d912ef-951c-4eb8-9855-5a0155ff19e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:24:57.535579 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d912ef-951c-4eb8-9855-5a0155ff19e0 None admin] PATH_INFO: `/v3/services` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:24:57.536289 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46d912ef-951c-4eb8-9855-5a0155ff19e0 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:24:57.536599 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46d912ef-951c-4eb8-9855-5a0155ff19e0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:24:57.537997 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46d912ef-951c-4eb8-9855-5a0155ff19e0 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:24:57.539771 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d42bff-5fad-4d2b-be30-89ad32bc8012 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:24:57.540423 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d42bff-5fad-4d2b-be30-89ad32bc8012 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:24:57.540703 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d42bff-5fad-4d2b-be30-89ad32bc8012 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:24:57.541003 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d42bff-5fad-4d2b-be30-89ad32bc8012 None admin] PATH_INFO: `/v3/projects` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:24:57.542388 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10d42bff-5fad-4d2b-be30-89ad32bc8012 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:24:57.542826 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10d42bff-5fad-4d2b-be30-89ad32bc8012 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:24:57.548379 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 478/939] 10.222.0.33 () {56 vars in 1126 bytes} [Mon Jul 25 05:24:57 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:24:57.548553 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10d42bff-5fad-4d2b-be30-89ad32bc8012 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:24:57.548657 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d912ef-951c-4eb8-9855-5a0155ff19e0 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:24:57.560028 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 462/940] 10.222.0.33 () {56 vars in 1148 bytes} [Mon Jul 25 05:24:57 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:24:57.560859 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d42bff-5fad-4d2b-be30-89ad32bc8012 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:24:57.636013 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:24:57.659914 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d912ef-951c-4eb8-9855-5a0155ff19e0 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:24:57.674761 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d42bff-5fad-4d2b-be30-89ad32bc8012 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:24:57.689309 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:24:57.695593 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c2a7f7c-1cdc-4c30-b09f-11726d92f360 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:24:57.696256 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2a7f7c-1cdc-4c30-b09f-11726d92f360 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:24:57.696548 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2a7f7c-1cdc-4c30-b09f-11726d92f360 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:24:57.696848 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2a7f7c-1cdc-4c30-b09f-11726d92f360 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:24:57.697654 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c2a7f7c-1cdc-4c30-b09f-11726d92f360 None admin] RBAC: Adding query filter params (interface=public, service_id=f5c4b2feadf540eeb517ed0d942d220c, region_id=RegionOne) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:24:57.697983 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c2a7f7c-1cdc-4c30-b09f-11726d92f360 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:24:57.699463 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c2a7f7c-1cdc-4c30-b09f-11726d92f360 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:24:57.709323 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 479/941] 10.222.0.33 () {56 vars in 1251 bytes} [Mon Jul 25 05:24:57 2022] GET /identity/v3/endpoints?service_id=f5c4b2feadf540eeb517ed0d942d220c&interface=public®ion_id=RegionOne => generated 194 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:24:57.709323 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c2a7f7c-1cdc-4c30-b09f-11726d92f360 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:24:57.717769 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f7276ff-2bc0-42f6-b731-87df4417beef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:24:57.718836 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7276ff-2bc0-42f6-b731-87df4417beef None admin] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:24:57.719124 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7276ff-2bc0-42f6-b731-87df4417beef None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:24:57.719401 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7276ff-2bc0-42f6-b731-87df4417beef None admin] PATH_INFO: `/v3/endpoints` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:24:57.720490 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f7276ff-2bc0-42f6-b731-87df4417beef None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:24:57.722211 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f7276ff-2bc0-42f6-b731-87df4417beef None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:24:57.745347 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 463/942] 10.222.0.33 () {60 vars in 1144 bytes} [Mon Jul 25 05:24:57 2022] POST /identity/v3/endpoints => generated 347 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 05:24:57.747387 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f7276ff-2bc0-42f6-b731-87df4417beef None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:24:57.846645 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c2a7f7c-1cdc-4c30-b09f-11726d92f360 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:24:57.900326 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f7276ff-2bc0-42f6-b731-87df4417beef None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:24:57.908573 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a358aaa7-8919-4026-99c5-8af7095726da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:24:57.910079 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a358aaa7-8919-4026-99c5-8af7095726da None admin] REQUEST_METHOD: `PUT` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:24:57.910370 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a358aaa7-8919-4026-99c5-8af7095726da None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:24:57.911660 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a358aaa7-8919-4026-99c5-8af7095726da None admin] PATH_INFO: `/v3/projects/92afba266cce4d1bad4134c351e74be3/users/3352531be1b54f76b864383ccc2e5ec2/roles/e85cbdc7a77a4fc29d71c8a3ae0f56f4` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:24:57.913353 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a358aaa7-8919-4026-99c5-8af7095726da None admin] RBAC: Authorizing `identity:create_grant(project_id=92afba266cce4d1bad4134c351e74be3, user_id=3352531be1b54f76b864383ccc2e5ec2, role_id=e85cbdc7a77a4fc29d71c8a3ae0f56f4)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:24:57.915139 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a358aaa7-8919-4026-99c5-8af7095726da None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:24:57.932303 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:24:57.941901 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 480/943] 10.222.0.33 () {58 vars in 1439 bytes} [Mon Jul 25 05:24:57 2022] PUT /identity/v3/projects/92afba266cce4d1bad4134c351e74be3/users/3352531be1b54f76b864383ccc2e5ec2/roles/e85cbdc7a77a4fc29d71c8a3ae0f56f4 => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 05:24:57.946876 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45748526-1b8d-4782-829a-079d04c83ef1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:24:57.947977 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45748526-1b8d-4782-829a-079d04c83ef1 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:24:57.949242 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45748526-1b8d-4782-829a-079d04c83ef1 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:24:57.949731 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45748526-1b8d-4782-829a-079d04c83ef1 None admin] PATH_INFO: `/v3/services` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:24:57.950653 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45748526-1b8d-4782-829a-079d04c83ef1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:24:57.952288 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45748526-1b8d-4782-829a-079d04c83ef1 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:24:57.959852 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 464/944] 10.222.0.33 () {56 vars in 1087 bytes} [Mon Jul 25 05:24:57 2022] GET /identity/v3/services => generated 1497 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 05:24:58.350735 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7647716c-70d9-41cb-9add-66ec1cba208f None None] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:24:58.351287 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7647716c-70d9-41cb-9add-66ec1cba208f None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:24:58.351287 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7647716c-70d9-41cb-9add-66ec1cba208f None None] PATH_INFO: `/` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:24:58.352990 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 481/945] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jul 25 05:24:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:24:58.358522 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4826c6e8-b874-4e3c-9e0d-8745338ceed0 None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:24:58.358973 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4826c6e8-b874-4e3c-9e0d-8745338ceed0 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:24:58.359178 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4826c6e8-b874-4e3c-9e0d-8745338ceed0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:24:58.376651 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-4826c6e8-b874-4e3c-9e0d-8745338ceed0 None None] MFA Rules not processed for user `1eae540242d94c10b0ad33960a855f08`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:24:58.377887 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-4826c6e8-b874-4e3c-9e0d-8745338ceed0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:24:58.406489 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:24:58.458147 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 465/946] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jul 25 05:24:58 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 25 05:24:58.482375 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7647716c-70d9-41cb-9add-66ec1cba208f None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:24:58.596068 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7647716c-70d9-41cb-9add-66ec1cba208f None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:24:58.630721 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c4659e-f811-4b06-aa11-8b54f5cbb07b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:24:58.631311 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c4659e-f811-4b06-aa11-8b54f5cbb07b None admin] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:24:58.631598 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c4659e-f811-4b06-aa11-8b54f5cbb07b None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:24:58.631878 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c4659e-f811-4b06-aa11-8b54f5cbb07b None admin] PATH_INFO: `/v3/projects` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:24:58.632600 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02c4659e-f811-4b06-aa11-8b54f5cbb07b None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:24:58.634068 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02c4659e-f811-4b06-aa11-8b54f5cbb07b None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:24:58.635963 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: WARNING keystone.server.flask.common [None req-02c4659e-f811-4b06-aa11-8b54f5cbb07b None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Jul 25 05:24:58.658240 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.sql.core [None req-02c4659e-f811-4b06-aa11-8b54f5cbb07b None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jul 25 05:24:58.658240 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Jul 25 05:24:58.658240 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [parameters: {'id': '9a929a7f573a42858e246055007b87ec', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Jul 25 05:24:58.658240 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80249) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jul 25 05:24:58.675436 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application [None req-02c4659e-f811-4b06-aa11-8b54f5cbb07b None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 9a929a7f573a42858e246055007b87ec.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 9a929a7f573a42858e246055007b87ec. Jul 25 05:24:58.675436 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:24:58.675436 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 25 05:24:58.675436 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 25 05:24:58.675436 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 25 05:24:58.675436 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 25 05:24:58.675436 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 25 05:24:58.675436 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application result = self._query(query) Jul 25 05:24:58.675436 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 25 05:24:58.675436 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application conn.query(q) Jul 25 05:24:58.675436 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 25 05:24:58.675436 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 25 05:24:58.675436 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 25 05:24:58.676301 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application result.read() Jul 25 05:24:58.676301 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 25 05:24:58.676301 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 25 05:24:58.676301 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 25 05:24:58.676301 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 25 05:24:58.676301 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 25 05:24:58.676301 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 25 05:24:58.676301 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 25 05:24:58.676301 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 25 05:24:58.676301 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jul 25 05:24:58.676301 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Jul 25 05:24:58.676301 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jul 25 05:24:58.676301 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Jul 25 05:24:58.676301 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:24:58.676301 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jul 25 05:24:58.676301 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jul 25 05:24:58.676301 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Jul 25 05:24:58.676301 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return project_ref.to_dict() Jul 25 05:24:58.677779 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 25 05:24:58.677779 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application next(self.gen) Jul 25 05:24:58.677779 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jul 25 05:24:58.677779 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application yield resource Jul 25 05:24:58.677779 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 25 05:24:58.677779 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application next(self.gen) Jul 25 05:24:58.677779 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jul 25 05:24:58.677779 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application self.session.rollback() Jul 25 05:24:58.677779 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jul 25 05:24:58.677779 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application self.force_reraise() Jul 25 05:24:58.677779 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jul 25 05:24:58.677779 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise self.value Jul 25 05:24:58.677779 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jul 25 05:24:58.677779 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jul 25 05:24:58.677779 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jul 25 05:24:58.677779 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application session.commit() Jul 25 05:24:58.677779 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jul 25 05:24:58.678956 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jul 25 05:24:58.678956 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jul 25 05:24:58.678956 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application self._prepare_impl() Jul 25 05:24:58.678956 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jul 25 05:24:58.678956 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application self.session.flush() Jul 25 05:24:58.678956 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jul 25 05:24:58.678956 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application self._flush(objects) Jul 25 05:24:58.678956 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jul 25 05:24:58.678956 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jul 25 05:24:58.678956 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jul 25 05:24:58.678956 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application compat.raise_( Jul 25 05:24:58.678956 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 25 05:24:58.678956 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception Jul 25 05:24:58.678956 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jul 25 05:24:58.678956 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application flush_context.execute() Jul 25 05:24:58.678956 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jul 25 05:24:58.678956 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application rec.execute(self) Jul 25 05:24:58.679907 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jul 25 05:24:58.679907 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jul 25 05:24:58.679907 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jul 25 05:24:58.679907 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application _emit_insert_statements( Jul 25 05:24:58.679907 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Jul 25 05:24:58.679907 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application c = connection._execute_20( Jul 25 05:24:58.679907 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jul 25 05:24:58.679907 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jul 25 05:24:58.679907 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Jul 25 05:24:58.679907 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jul 25 05:24:58.679907 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jul 25 05:24:58.679907 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application ret = self._execute_context( Jul 25 05:24:58.679907 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jul 25 05:24:58.679907 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jul 25 05:24:58.679907 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jul 25 05:24:58.680732 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jul 25 05:24:58.680732 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 25 05:24:58.680732 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception Jul 25 05:24:58.680732 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 25 05:24:58.680732 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 25 05:24:58.680732 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 25 05:24:58.680732 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 25 05:24:58.680732 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 25 05:24:58.680732 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application result = self._query(query) Jul 25 05:24:58.680732 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 25 05:24:58.680732 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application conn.query(q) Jul 25 05:24:58.680732 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 25 05:24:58.680732 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 25 05:24:58.680732 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 25 05:24:58.680732 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application result.read() Jul 25 05:24:58.680732 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 25 05:24:58.680732 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 25 05:24:58.683512 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 25 05:24:58.683512 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 25 05:24:58.683512 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 25 05:24:58.683512 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 25 05:24:58.683512 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 25 05:24:58.683512 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 25 05:24:58.683512 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jul 25 05:24:58.683512 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Jul 25 05:24:58.683512 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application [parameters: {'id': '9a929a7f573a42858e246055007b87ec', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Jul 25 05:24:58.683512 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jul 25 05:24:58.683512 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Jul 25 05:24:58.683512 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 25 05:24:58.683512 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Jul 25 05:24:58.683512 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:24:58.683512 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Jul 25 05:24:58.685641 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Jul 25 05:24:58.685641 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jul 25 05:24:58.685641 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jul 25 05:24:58.685641 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Jul 25 05:24:58.685641 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Jul 25 05:24:58.685641 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 25 05:24:58.685641 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Jul 25 05:24:58.685641 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:24:58.685641 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:24:58.685641 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:24:58.685641 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:24:58.685641 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:24:58.685641 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:24:58.685641 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:24:58.685641 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:24:58.685641 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:24:58.685641 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:24:58.692853 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:24:58.692853 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Jul 25 05:24:58.692853 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Jul 25 05:24:58.692853 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:24:58.692853 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:24:58.692853 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Jul 25 05:24:58.692853 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.Conflict( Jul 25 05:24:58.692853 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 9a929a7f573a42858e246055007b87ec. Jul 25 05:24:58.692853 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Jul 25 05:24:58.692853 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 482/947] 10.222.0.33 () {60 vars in 1140 bytes} [Mon Jul 25 05:24:58 2022] POST /identity/v3/projects => generated 262 bytes in 195 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:24:58.695787 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4826c6e8-b874-4e3c-9e0d-8745338ceed0 None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:24:58.801872 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4826c6e8-b874-4e3c-9e0d-8745338ceed0 None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:24:58.837777 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59fac049-ff79-4cef-b717-df973a4ca267 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:24:58.838602 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59fac049-ff79-4cef-b717-df973a4ca267 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:24:58.839120 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59fac049-ff79-4cef-b717-df973a4ca267 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:24:58.839401 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59fac049-ff79-4cef-b717-df973a4ca267 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:24:58.846259 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59fac049-ff79-4cef-b717-df973a4ca267 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:24:58.847704 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59fac049-ff79-4cef-b717-df973a4ca267 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:24:58.862019 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application [None req-59fac049-ff79-4cef-b717-df973a4ca267 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 05:24:58.862019 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:24:58.862019 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:24:58.862019 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:24:58.862019 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:24:58.862019 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:24:58.862019 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:24:58.862019 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:24:58.862019 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:24:58.862019 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:24:58.862019 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:24:58.862019 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:24:58.862019 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 25 05:24:58.862019 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 25 05:24:58.862019 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 25 05:24:58.864152 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 25 05:24:58.864152 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:24:58.864152 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:24:58.864152 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:24:58.864152 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:24:58.864152 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:24:58.864152 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:24:58.864152 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:24:58.864152 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application with Lock( Jul 25 05:24:58.864152 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:24:58.864152 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:24:58.864152 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:24:58.864152 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:24:58.864152 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:24:58.864152 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:24:58.864152 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:24:58.864152 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:24:58.866515 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 25 05:24:58.866515 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 25 05:24:58.866515 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 25 05:24:58.866515 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 25 05:24:58.866515 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 25 05:24:58.866515 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 25 05:24:58.866515 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 05:24:58.866515 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Jul 25 05:24:58.866515 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 466/948] 10.222.0.33 () {56 vars in 1111 bytes} [Mon Jul 25 05:24:58 2022] GET /identity/v3/projects/service => generated 88 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:24:58.874385 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c4659e-f811-4b06-aa11-8b54f5cbb07b None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:24:58.935029 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:24:58.985233 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c4659e-f811-4b06-aa11-8b54f5cbb07b None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:24:59.020673 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d15c4c-7b5a-4695-9c3f-ab4a96f5e5fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:24:59.021381 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d15c4c-7b5a-4695-9c3f-ab4a96f5e5fd None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:24:59.021668 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d15c4c-7b5a-4695-9c3f-ab4a96f5e5fd None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:24:59.021950 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d15c4c-7b5a-4695-9c3f-ab4a96f5e5fd None admin] PATH_INFO: `/v3/projects` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:24:59.022719 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6d15c4c-7b5a-4695-9c3f-ab4a96f5e5fd None admin] RBAC: Adding query filter params (name=service) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:24:59.023029 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6d15c4c-7b5a-4695-9c3f-ab4a96f5e5fd None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:24:59.024763 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6d15c4c-7b5a-4695-9c3f-ab4a96f5e5fd None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:24:59.046477 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 483/949] 10.222.0.33 () {56 vars in 1112 bytes} [Mon Jul 25 05:24:58 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:24:59.902395 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee36176a-cc29-4f7c-b6f1-f4484d533164 None None] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:24:59.902953 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee36176a-cc29-4f7c-b6f1-f4484d533164 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:24:59.902953 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee36176a-cc29-4f7c-b6f1-f4484d533164 None None] PATH_INFO: `/` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:24:59.904671 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 467/950] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jul 25 05:24:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:24:59.910406 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df27c267-9cc7-435d-8c3c-b786971e0af7 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:24:59.910828 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df27c267-9cc7-435d-8c3c-b786971e0af7 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:24:59.911037 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df27c267-9cc7-435d-8c3c-b786971e0af7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:24:59.928242 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-df27c267-9cc7-435d-8c3c-b786971e0af7 None None] MFA Rules not processed for user `1eae540242d94c10b0ad33960a855f08`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:24:59.929796 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-df27c267-9cc7-435d-8c3c-b786971e0af7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:25:00.000805 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 484/951] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jul 25 05:24:59 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 25 05:25:00.023209 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee36176a-cc29-4f7c-b6f1-f4484d533164 None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:25:00.049127 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:25:00.101344 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7901d80-6747-4043-a405-06a5ba3d3f2b None None] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:25:00.101541 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7901d80-6747-4043-a405-06a5ba3d3f2b None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:25:00.101829 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7901d80-6747-4043-a405-06a5ba3d3f2b None None] PATH_INFO: `/` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:25:00.103482 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 485/952] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jul 25 05:25:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:25:00.109329 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bde32e2-31db-45cf-96e8-cf077659802b None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:25:00.109740 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bde32e2-31db-45cf-96e8-cf077659802b None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:25:00.110023 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bde32e2-31db-45cf-96e8-cf077659802b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:25:00.127996 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-2bde32e2-31db-45cf-96e8-cf077659802b None None] MFA Rules not processed for user `1eae540242d94c10b0ad33960a855f08`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:25:00.129221 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-2bde32e2-31db-45cf-96e8-cf077659802b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:25:00.139879 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee36176a-cc29-4f7c-b6f1-f4484d533164 None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:25:00.176487 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c31324ca-3aab-42b9-bf8c-1458f1e17c99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:25:00.176740 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:25:00.177322 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31324ca-3aab-42b9-bf8c-1458f1e17c99 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:25:00.177615 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31324ca-3aab-42b9-bf8c-1458f1e17c99 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:25:00.177895 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31324ca-3aab-42b9-bf8c-1458f1e17c99 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:25:00.178555 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c31324ca-3aab-42b9-bf8c-1458f1e17c99 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:25:00.179952 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c31324ca-3aab-42b9-bf8c-1458f1e17c99 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:25:00.192812 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application [None req-c31324ca-3aab-42b9-bf8c-1458f1e17c99 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jul 25 05:25:00.192812 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:25:00.192812 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 25 05:25:00.192812 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 25 05:25:00.192812 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 25 05:25:00.192812 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 25 05:25:00.192812 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 25 05:25:00.192812 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 25 05:25:00.192812 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 25 05:25:00.192812 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Jul 25 05:25:00.192812 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 25 05:25:00.192812 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Jul 25 05:25:00.192812 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:25:00.192812 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:25:00.192812 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:25:00.192812 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:25:00.192812 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:25:00.193912 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:25:00.193912 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:25:00.193912 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:25:00.193912 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:25:00.193912 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:25:00.193912 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:25:00.193912 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 25 05:25:00.193912 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 25 05:25:00.193912 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 25 05:25:00.193912 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 25 05:25:00.193912 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:25:00.193912 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:25:00.193912 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:25:00.193912 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:25:00.193912 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:25:00.193912 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:25:00.194786 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:25:00.194786 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application with Lock( Jul 25 05:25:00.194786 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:25:00.194786 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:25:00.194786 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:25:00.194786 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:25:00.194786 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:25:00.194786 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:25:00.194786 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:25:00.194786 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:25:00.194786 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 25 05:25:00.194786 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 25 05:25:00.194786 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 25 05:25:00.194786 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Jul 25 05:25:00.194786 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 468/953] 10.222.0.33 () {56 vars in 1111 bytes} [Mon Jul 25 05:25:00 2022] GET /identity/v3/services/compute => generated 88 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:25:00.195626 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f40435f-bca2-4b49-b409-cf1b8bd5ac08 None None] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:25:00.196283 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f40435f-bca2-4b49-b409-cf1b8bd5ac08 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:25:00.196283 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f40435f-bca2-4b49-b409-cf1b8bd5ac08 None None] PATH_INFO: `/` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:25:00.200240 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 469/954] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jul 25 05:25:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:25:00.200240 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f40435f-bca2-4b49-b409-cf1b8bd5ac08 None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:25:00.211656 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 486/955] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jul 25 05:25:00 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 25 05:25:00.221313 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793e0c51-f042-45ec-ba2a-02695f21ac03 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:25:00.221603 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793e0c51-f042-45ec-ba2a-02695f21ac03 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:25:00.221883 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793e0c51-f042-45ec-ba2a-02695f21ac03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:25:00.242135 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-793e0c51-f042-45ec-ba2a-02695f21ac03 None None] MFA Rules not processed for user `1eae540242d94c10b0ad33960a855f08`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:25:00.243425 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-793e0c51-f042-45ec-ba2a-02695f21ac03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:25:00.290978 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:25:00.313418 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f40435f-bca2-4b49-b409-cf1b8bd5ac08 None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:25:00.332820 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 487/956] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jul 25 05:25:00 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 25 05:25:00.332820 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-793e0c51-f042-45ec-ba2a-02695f21ac03 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:25:00.353731 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd1fcc5c-094b-4e76-b37a-45089eee4bf3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:25:00.354829 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1fcc5c-094b-4e76-b37a-45089eee4bf3 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:25:00.355123 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1fcc5c-094b-4e76-b37a-45089eee4bf3 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:25:00.355402 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1fcc5c-094b-4e76-b37a-45089eee4bf3 None admin] PATH_INFO: `/v3/services` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:25:00.356148 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd1fcc5c-094b-4e76-b37a-45089eee4bf3 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:25:00.356457 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd1fcc5c-094b-4e76-b37a-45089eee4bf3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:25:00.358067 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd1fcc5c-094b-4e76-b37a-45089eee4bf3 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:25:00.363283 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 470/957] 10.222.0.33 () {56 vars in 1112 bytes} [Mon Jul 25 05:25:00 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 05:25:00.363918 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd1fcc5c-094b-4e76-b37a-45089eee4bf3 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:25:00.443083 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-793e0c51-f042-45ec-ba2a-02695f21ac03 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:25:00.451024 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:25:00.470864 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd1fcc5c-094b-4e76-b37a-45089eee4bf3 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:25:00.479214 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e995b47-c550-4e0c-b5d9-2cf60d10e272 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:25:00.480122 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e995b47-c550-4e0c-b5d9-2cf60d10e272 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:25:00.480410 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e995b47-c550-4e0c-b5d9-2cf60d10e272 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:25:00.480689 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e995b47-c550-4e0c-b5d9-2cf60d10e272 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:25:00.481440 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e995b47-c550-4e0c-b5d9-2cf60d10e272 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:25:00.482810 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e995b47-c550-4e0c-b5d9-2cf60d10e272 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:25:00.494654 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application [None req-1e995b47-c550-4e0c-b5d9-2cf60d10e272 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 25 05:25:00.494654 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:25:00.494654 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 25 05:25:00.494654 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 25 05:25:00.494654 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 25 05:25:00.494654 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 25 05:25:00.494654 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 25 05:25:00.494654 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 25 05:25:00.494654 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 25 05:25:00.494654 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Jul 25 05:25:00.494654 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 25 05:25:00.494654 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Jul 25 05:25:00.494654 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:25:00.494654 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:25:00.494654 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:25:00.494654 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:25:00.494654 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:25:00.496514 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:25:00.496514 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:25:00.496514 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:25:00.496514 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:25:00.496514 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:25:00.496514 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:25:00.496514 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 25 05:25:00.496514 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 25 05:25:00.496514 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 25 05:25:00.496514 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 25 05:25:00.496514 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:25:00.496514 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:25:00.496514 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:25:00.496514 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:25:00.496514 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:25:00.496514 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:25:00.497424 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:25:00.497424 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application with Lock( Jul 25 05:25:00.497424 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:25:00.497424 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:25:00.497424 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:25:00.497424 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:25:00.497424 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:25:00.497424 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:25:00.497424 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:25:00.497424 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:25:00.497424 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 25 05:25:00.497424 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 25 05:25:00.497424 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 25 05:25:00.497424 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Jul 25 05:25:00.497424 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 488/958] 10.222.0.33 () {56 vars in 1126 bytes} [Mon Jul 25 05:25:00 2022] GET /identity/v3/services/object-store => generated 93 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:25:00.498245 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e995b47-c550-4e0c-b5d9-2cf60d10e272 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:25:00.514192 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb8b265-b993-4117-8c4a-671809f8c603 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:25:00.515002 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb8b265-b993-4117-8c4a-671809f8c603 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:25:00.515284 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb8b265-b993-4117-8c4a-671809f8c603 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:25:00.515556 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb8b265-b993-4117-8c4a-671809f8c603 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:25:00.526278 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fb8b265-b993-4117-8c4a-671809f8c603 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 05:25:00.534335 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application [None req-5fb8b265-b993-4117-8c4a-671809f8c603 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 25 05:25:00.534335 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:25:00.534335 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:25:00.534335 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:25:00.534335 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:25:00.534335 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:25:00.534335 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:25:00.534335 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:25:00.534335 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:25:00.534335 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:25:00.534335 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:25:00.534335 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:25:00.534335 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 05:25:00.534335 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 05:25:00.534335 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 05:25:00.534335 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 05:25:00.535300 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 05:25:00.535300 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 05:25:00.535300 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 05:25:00.535300 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 05:25:00.535300 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:25:00.535300 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:25:00.535300 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:25:00.535300 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:25:00.535300 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:25:00.535300 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:25:00.535300 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:25:00.535300 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application with Lock( Jul 25 05:25:00.535300 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:25:00.535300 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:25:00.535300 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:25:00.535300 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:25:00.536184 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:25:00.536184 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:25:00.536184 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:25:00.536184 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:25:00.536184 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 05:25:00.536184 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 05:25:00.536184 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 05:25:00.536184 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 05:25:00.536184 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 05:25:00.536184 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 05:25:00.536184 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 25 05:25:00.536184 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Jul 25 05:25:00.536184 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 471/959] 10.222.0.33 () {56 vars in 1120 bytes} [Mon Jul 25 05:25:00 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:25:00.536987 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb8b265-b993-4117-8c4a-671809f8c603 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:25:00.546684 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:25:00.613711 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e995b47-c550-4e0c-b5d9-2cf60d10e272 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:25:00.646402 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30488354-c734-453e-9cc0-52e5ff3b669d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:25:00.647067 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30488354-c734-453e-9cc0-52e5ff3b669d None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:25:00.647353 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30488354-c734-453e-9cc0-52e5ff3b669d None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:25:00.647631 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30488354-c734-453e-9cc0-52e5ff3b669d None admin] PATH_INFO: `/v3/services` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:25:00.648344 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30488354-c734-453e-9cc0-52e5ff3b669d None admin] RBAC: Adding query filter params (type=compute) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:25:00.648650 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30488354-c734-453e-9cc0-52e5ff3b669d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:25:00.650041 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30488354-c734-453e-9cc0-52e5ff3b669d None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:25:00.650599 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb8b265-b993-4117-8c4a-671809f8c603 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:25:00.655125 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 489/960] 10.222.0.33 () {56 vars in 1112 bytes} [Mon Jul 25 05:25:00 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:25:00.655905 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30488354-c734-453e-9cc0-52e5ff3b669d None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:25:00.704149 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-febbe26e-7e4e-4cbe-8da3-f3cedfcc008c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:25:00.705225 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:25:00.707394 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-febbe26e-7e4e-4cbe-8da3-f3cedfcc008c None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:25:00.708024 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-febbe26e-7e4e-4cbe-8da3-f3cedfcc008c None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:25:00.708488 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-febbe26e-7e4e-4cbe-8da3-f3cedfcc008c None admin] PATH_INFO: `/v3/services` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:25:00.709452 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-febbe26e-7e4e-4cbe-8da3-f3cedfcc008c None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:25:00.709955 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-febbe26e-7e4e-4cbe-8da3-f3cedfcc008c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:25:00.711641 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-febbe26e-7e4e-4cbe-8da3-f3cedfcc008c None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:25:00.725360 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 472/961] 10.222.0.33 () {56 vars in 1122 bytes} [Mon Jul 25 05:25:00 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:25:00.727444 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-febbe26e-7e4e-4cbe-8da3-f3cedfcc008c None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:25:00.805117 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30488354-c734-453e-9cc0-52e5ff3b669d None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:25:00.821064 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:25:00.861731 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d37a40d5-d591-41b6-b5f3-ca93afedcca7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:25:00.863214 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37a40d5-d591-41b6-b5f3-ca93afedcca7 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:25:00.864036 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37a40d5-d591-41b6-b5f3-ca93afedcca7 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:25:00.864511 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37a40d5-d591-41b6-b5f3-ca93afedcca7 None admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:25:00.865496 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d37a40d5-d591-41b6-b5f3-ca93afedcca7 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:25:00.865671 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-febbe26e-7e4e-4cbe-8da3-f3cedfcc008c None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:25:00.866157 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d37a40d5-d591-41b6-b5f3-ca93afedcca7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:25:00.867891 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d37a40d5-d591-41b6-b5f3-ca93afedcca7 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:25:00.879725 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 490/962] 10.222.0.33 () {56 vars in 1115 bytes} [Mon Jul 25 05:25:00 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:25:00.884865 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d37a40d5-d591-41b6-b5f3-ca93afedcca7 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:25:00.911494 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5798f391-b8d8-4f41-a44c-1dad760e361e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:25:00.913076 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5798f391-b8d8-4f41-a44c-1dad760e361e None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:25:00.913734 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5798f391-b8d8-4f41-a44c-1dad760e361e None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:25:00.914194 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5798f391-b8d8-4f41-a44c-1dad760e361e None admin] PATH_INFO: `/v3/services` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:25:00.915671 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5798f391-b8d8-4f41-a44c-1dad760e361e None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:25:00.916167 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5798f391-b8d8-4f41-a44c-1dad760e361e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:25:00.918301 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5798f391-b8d8-4f41-a44c-1dad760e361e None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:25:00.925881 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 473/963] 10.222.0.33 () {56 vars in 1122 bytes} [Mon Jul 25 05:25:00 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 05:25:00.931172 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5798f391-b8d8-4f41-a44c-1dad760e361e None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:25:01.000695 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:25:01.008910 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d37a40d5-d591-41b6-b5f3-ca93afedcca7 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:25:01.051744 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf3a637a-3052-4fcf-b380-d9596cf79881 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:25:01.052522 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5798f391-b8d8-4f41-a44c-1dad760e361e None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:25:01.053046 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3a637a-3052-4fcf-b380-d9596cf79881 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:25:01.053490 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3a637a-3052-4fcf-b380-d9596cf79881 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:25:01.053920 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3a637a-3052-4fcf-b380-d9596cf79881 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:25:01.068256 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf3a637a-3052-4fcf-b380-d9596cf79881 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:25:01.070612 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf3a637a-3052-4fcf-b380-d9596cf79881 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:25:01.090056 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-068726e2-64fb-4c12-b48a-fc8de40e9190 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:25:01.090850 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068726e2-64fb-4c12-b48a-fc8de40e9190 None admin] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:25:01.091228 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068726e2-64fb-4c12-b48a-fc8de40e9190 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:25:01.091647 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068726e2-64fb-4c12-b48a-fc8de40e9190 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:25:01.092483 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-068726e2-64fb-4c12-b48a-fc8de40e9190 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:25:01.093607 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application [None req-cf3a637a-3052-4fcf-b380-d9596cf79881 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 05:25:01.093607 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:25:01.093607 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:25:01.093607 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:25:01.093607 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:25:01.093607 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:25:01.093607 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:25:01.093607 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:25:01.093607 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:25:01.093607 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:25:01.093607 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:25:01.093607 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:25:01.093607 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 05:25:01.093607 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 05:25:01.093607 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 05:25:01.094692 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 05:25:01.094692 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:25:01.094692 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:25:01.094692 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:25:01.094692 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:25:01.094692 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:25:01.094692 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:25:01.094692 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:25:01.094692 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application with Lock( Jul 25 05:25:01.094692 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:25:01.094692 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:25:01.094692 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:25:01.094692 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:25:01.094692 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:25:01.094692 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:25:01.094692 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:25:01.094692 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:25:01.095820 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 05:25:01.095820 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 05:25:01.095820 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 05:25:01.095820 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Jul 25 05:25:01.096151 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-068726e2-64fb-4c12-b48a-fc8de40e9190 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:25:01.097857 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 491/964] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Jul 25 05:25:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 25 05:25:01.103318 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf3a637a-3052-4fcf-b380-d9596cf79881 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:25:01.105200 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:25:01.128904 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 474/965] 10.222.0.33 () {60 vars in 1144 bytes} [Mon Jul 25 05:25:00 2022] POST /identity/v3/endpoints => generated 349 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 05:25:01.213046 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fdbca52-54c3-4fd7-a170-9cd7e6224081 None None] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:25:01.213420 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fdbca52-54c3-4fd7-a170-9cd7e6224081 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:25:01.213758 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fdbca52-54c3-4fd7-a170-9cd7e6224081 None None] PATH_INFO: `/` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:25:01.215686 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 475/966] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jul 25 05:25:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 25 05:25:01.221675 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd33704-0007-4fb4-950b-a4dec57ed965 None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:25:01.222410 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd33704-0007-4fb4-950b-a4dec57ed965 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:25:01.223015 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd33704-0007-4fb4-950b-a4dec57ed965 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:25:01.229257 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf3a637a-3052-4fcf-b380-d9596cf79881 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:25:01.241989 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-afd33704-0007-4fb4-950b-a4dec57ed965 None None] MFA Rules not processed for user `1eae540242d94c10b0ad33960a855f08`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:25:01.243759 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-afd33704-0007-4fb4-950b-a4dec57ed965 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:25:01.273083 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a31388e-4eee-4963-a04a-ecf2c86a6bf3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:25:01.273875 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a31388e-4eee-4963-a04a-ecf2c86a6bf3 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:25:01.274161 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a31388e-4eee-4963-a04a-ecf2c86a6bf3 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:25:01.274440 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a31388e-4eee-4963-a04a-ecf2c86a6bf3 None admin] PATH_INFO: `/v3/domains` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:25:01.275299 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a31388e-4eee-4963-a04a-ecf2c86a6bf3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:25:01.275610 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a31388e-4eee-4963-a04a-ecf2c86a6bf3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:25:01.277592 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a31388e-4eee-4963-a04a-ecf2c86a6bf3 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:25:01.291655 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 492/967] 10.222.0.33 () {56 vars in 1109 bytes} [Mon Jul 25 05:25:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:25:01.298038 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a31388e-4eee-4963-a04a-ecf2c86a6bf3 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:25:01.302222 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:25:01.333562 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 476/968] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jul 25 05:25:01 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 25 05:25:01.361152 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd33704-0007-4fb4-950b-a4dec57ed965 None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:25:01.434179 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a31388e-4eee-4963-a04a-ecf2c86a6bf3 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:25:01.474259 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d64b9dff-e49b-436c-8852-2c4979e433e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:25:01.475172 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64b9dff-e49b-436c-8852-2c4979e433e0 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:25:01.475529 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64b9dff-e49b-436c-8852-2c4979e433e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:25:01.475882 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64b9dff-e49b-436c-8852-2c4979e433e0 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:25:01.481319 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd33704-0007-4fb4-950b-a4dec57ed965 None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:25:01.483780 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d64b9dff-e49b-436c-8852-2c4979e433e0 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:25:01.485340 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d64b9dff-e49b-436c-8852-2c4979e433e0 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:25:01.501408 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application [None req-d64b9dff-e49b-436c-8852-2c4979e433e0 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 25 05:25:01.501408 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:25:01.501408 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:25:01.501408 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:25:01.501408 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:25:01.501408 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:25:01.501408 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:25:01.501408 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:25:01.501408 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:25:01.501408 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:25:01.501408 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:25:01.501408 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:25:01.501408 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 25 05:25:01.501408 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 25 05:25:01.501408 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 25 05:25:01.501408 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 25 05:25:01.502464 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:25:01.502464 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:25:01.502464 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 25 05:25:01.502464 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 05:25:01.502464 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 25 05:25:01.502464 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 05:25:01.502464 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:25:01.502464 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:25:01.502464 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:25:01.502464 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:25:01.502464 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:25:01.502464 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application with Lock( Jul 25 05:25:01.502464 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:25:01.502464 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:25:01.502464 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:25:01.502464 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:25:01.502464 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:25:01.502464 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:25:01.503454 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:25:01.503454 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:25:01.503454 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 25 05:25:01.503454 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 25 05:25:01.503454 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 25 05:25:01.503454 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 25 05:25:01.503454 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 25 05:25:01.503454 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 25 05:25:01.503454 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 25 05:25:01.503454 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Jul 25 05:25:01.503454 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 493/969] 10.222.0.33 () {56 vars in 1117 bytes} [Mon Jul 25 05:25:01 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 25 05:25:01.509406 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:25:01.511960 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d64b9dff-e49b-436c-8852-2c4979e433e0 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:25:01.529159 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac0a0bb-f702-40c8-a334-c1b21e9ff4fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:25:01.530178 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac0a0bb-f702-40c8-a334-c1b21e9ff4fd None admin] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:25:01.530662 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac0a0bb-f702-40c8-a334-c1b21e9ff4fd None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:25:01.531135 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac0a0bb-f702-40c8-a334-c1b21e9ff4fd None admin] PATH_INFO: `/v3/users` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:25:01.532058 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ac0a0bb-f702-40c8-a334-c1b21e9ff4fd None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:25:01.533774 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ac0a0bb-f702-40c8-a334-c1b21e9ff4fd None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:25:01.535280 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: WARNING keystone.server.flask.common [None req-9ac0a0bb-f702-40c8-a334-c1b21e9ff4fd None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Jul 25 05:25:01.550485 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.sql.core [None req-9ac0a0bb-f702-40c8-a334-c1b21e9ff4fd None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jul 25 05:25:01.550485 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Jul 25 05:25:01.550485 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [parameters: {'user_id': '3cb896c920904bc68b41464bcc6997d2', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Jul 25 05:25:01.550485 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80248) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jul 25 05:25:01.562631 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application [None req-9ac0a0bb-f702-40c8-a334-c1b21e9ff4fd None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Jul 25 05:25:01.562631 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:25:01.562631 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 25 05:25:01.562631 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 25 05:25:01.562631 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 25 05:25:01.562631 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 25 05:25:01.562631 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 25 05:25:01.562631 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application result = self._query(query) Jul 25 05:25:01.562631 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 25 05:25:01.562631 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application conn.query(q) Jul 25 05:25:01.562631 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 25 05:25:01.562631 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 25 05:25:01.562631 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 25 05:25:01.562631 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application result.read() Jul 25 05:25:01.562631 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 25 05:25:01.563548 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 25 05:25:01.563548 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 25 05:25:01.563548 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 25 05:25:01.563548 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 25 05:25:01.563548 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 25 05:25:01.563548 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 25 05:25:01.563548 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 25 05:25:01.563548 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jul 25 05:25:01.563548 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Jul 25 05:25:01.563548 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jul 25 05:25:01.563548 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Jul 25 05:25:01.563548 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:25:01.563548 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jul 25 05:25:01.563548 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jul 25 05:25:01.563548 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Jul 25 05:25:01.563548 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Jul 25 05:25:01.563548 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 25 05:25:01.563548 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application next(self.gen) Jul 25 05:25:01.565221 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jul 25 05:25:01.565221 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application yield resource Jul 25 05:25:01.565221 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 25 05:25:01.565221 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application next(self.gen) Jul 25 05:25:01.565221 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jul 25 05:25:01.565221 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application self.session.rollback() Jul 25 05:25:01.565221 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jul 25 05:25:01.565221 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application self.force_reraise() Jul 25 05:25:01.565221 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jul 25 05:25:01.565221 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application raise self.value Jul 25 05:25:01.565221 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jul 25 05:25:01.565221 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jul 25 05:25:01.565221 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jul 25 05:25:01.565221 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application session.commit() Jul 25 05:25:01.565221 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jul 25 05:25:01.565221 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jul 25 05:25:01.565221 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jul 25 05:25:01.566275 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application self._prepare_impl() Jul 25 05:25:01.566275 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jul 25 05:25:01.566275 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application self.session.flush() Jul 25 05:25:01.566275 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jul 25 05:25:01.566275 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application self._flush(objects) Jul 25 05:25:01.566275 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jul 25 05:25:01.566275 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jul 25 05:25:01.566275 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jul 25 05:25:01.566275 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application compat.raise_( Jul 25 05:25:01.566275 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 25 05:25:01.566275 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application raise exception Jul 25 05:25:01.566275 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jul 25 05:25:01.566275 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application flush_context.execute() Jul 25 05:25:01.566275 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jul 25 05:25:01.566275 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application rec.execute(self) Jul 25 05:25:01.566275 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jul 25 05:25:01.566275 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jul 25 05:25:01.567190 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jul 25 05:25:01.567190 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application _emit_insert_statements( Jul 25 05:25:01.567190 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Jul 25 05:25:01.567190 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application result = connection._execute_20( Jul 25 05:25:01.567190 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jul 25 05:25:01.567190 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jul 25 05:25:01.567190 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Jul 25 05:25:01.567190 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jul 25 05:25:01.567190 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jul 25 05:25:01.567190 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application ret = self._execute_context( Jul 25 05:25:01.567190 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jul 25 05:25:01.567190 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jul 25 05:25:01.567190 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jul 25 05:25:01.567190 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jul 25 05:25:01.567190 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 25 05:25:01.567190 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application raise exception Jul 25 05:25:01.568062 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 25 05:25:01.568062 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 25 05:25:01.568062 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 25 05:25:01.568062 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 25 05:25:01.568062 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 25 05:25:01.568062 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application result = self._query(query) Jul 25 05:25:01.568062 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 25 05:25:01.568062 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application conn.query(q) Jul 25 05:25:01.568062 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 25 05:25:01.568062 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 25 05:25:01.568062 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 25 05:25:01.568062 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application result.read() Jul 25 05:25:01.568062 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 25 05:25:01.568062 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 25 05:25:01.568062 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 25 05:25:01.568062 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 25 05:25:01.570140 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 25 05:25:01.570140 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 25 05:25:01.570140 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 25 05:25:01.570140 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 25 05:25:01.570140 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jul 25 05:25:01.570140 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Jul 25 05:25:01.570140 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application [parameters: {'user_id': '3cb896c920904bc68b41464bcc6997d2', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Jul 25 05:25:01.570140 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jul 25 05:25:01.570140 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Jul 25 05:25:01.570140 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 25 05:25:01.570140 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Jul 25 05:25:01.570140 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:25:01.570140 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:25:01.570140 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:25:01.570140 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:25:01.572409 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:25:01.572409 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:25:01.572409 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:25:01.572409 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:25:01.572409 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:25:01.572409 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:25:01.572409 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:25:01.572409 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Jul 25 05:25:01.572409 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Jul 25 05:25:01.572409 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:25:01.572409 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:25:01.572409 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 25 05:25:01.572409 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 05:25:01.572409 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 25 05:25:01.572409 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 05:25:01.572409 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Jul 25 05:25:01.572409 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Jul 25 05:25:01.576228 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Jul 25 05:25:01.576228 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Jul 25 05:25:01.576228 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jul 25 05:25:01.576228 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jul 25 05:25:01.576228 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Jul 25 05:25:01.576228 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Jul 25 05:25:01.576228 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 477/970] 10.222.0.33 () {60 vars in 1131 bytes} [Mon Jul 25 05:25:01 2022] POST /identity/v3/users => generated 160 bytes in 211 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Jul 25 05:25:01.578362 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac0a0bb-f702-40c8-a334-c1b21e9ff4fd None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:25:01.599015 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:25:01.646724 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d64b9dff-e49b-436c-8852-2c4979e433e0 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:25:01.697560 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9289b870-879b-4c52-9fc0-90fa1e5ff25d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:25:01.699688 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9289b870-879b-4c52-9fc0-90fa1e5ff25d None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:25:01.700440 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9289b870-879b-4c52-9fc0-90fa1e5ff25d None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:25:01.701403 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9289b870-879b-4c52-9fc0-90fa1e5ff25d None admin] PATH_INFO: `/v3/users` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:25:01.702794 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9289b870-879b-4c52-9fc0-90fa1e5ff25d None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:25:01.703578 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9289b870-879b-4c52-9fc0-90fa1e5ff25d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:25:01.705844 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9289b870-879b-4c52-9fc0-90fa1e5ff25d None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:25:01.706821 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac0a0bb-f702-40c8-a334-c1b21e9ff4fd None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:25:01.739931 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 494/971] 10.222.0.33 () {56 vars in 1149 bytes} [Mon Jul 25 05:25:01 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:25:01.742263 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9289b870-879b-4c52-9fc0-90fa1e5ff25d None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:25:01.749165 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d2e6685-b49c-4017-b2bf-46346f753adb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:25:01.749961 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2e6685-b49c-4017-b2bf-46346f753adb None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:25:01.750245 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2e6685-b49c-4017-b2bf-46346f753adb None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:25:01.750523 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2e6685-b49c-4017-b2bf-46346f753adb None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:25:01.762779 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d2e6685-b49c-4017-b2bf-46346f753adb None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:25:01.765288 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d2e6685-b49c-4017-b2bf-46346f753adb None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:25:01.783613 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application [None req-2d2e6685-b49c-4017-b2bf-46346f753adb None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 25 05:25:01.783613 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:25:01.783613 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:25:01.783613 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:25:01.783613 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:25:01.783613 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:25:01.783613 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:25:01.783613 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:25:01.783613 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:25:01.783613 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:25:01.783613 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:25:01.783613 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:25:01.783613 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 25 05:25:01.783613 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 25 05:25:01.783613 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 25 05:25:01.783613 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 25 05:25:01.784843 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:25:01.784843 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:25:01.784843 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 25 05:25:01.784843 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 05:25:01.784843 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 25 05:25:01.784843 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 05:25:01.784843 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:25:01.784843 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:25:01.784843 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:25:01.784843 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:25:01.784843 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:25:01.784843 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application with Lock( Jul 25 05:25:01.784843 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:25:01.784843 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:25:01.784843 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:25:01.784843 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:25:01.784843 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:25:01.784843 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:25:01.785865 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:25:01.785865 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:25:01.785865 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 25 05:25:01.785865 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 25 05:25:01.785865 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 25 05:25:01.785865 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 25 05:25:01.785865 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 25 05:25:01.785865 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 25 05:25:01.785865 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 25 05:25:01.785865 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Jul 25 05:25:01.788329 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 478/972] 10.222.0.33 () {56 vars in 1099 bytes} [Mon Jul 25 05:25:01 2022] GET /identity/v3/users/cinder => generated 84 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 25 05:25:01.795237 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d2e6685-b49c-4017-b2bf-46346f753adb None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:25:01.857445 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:25:01.886448 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9289b870-879b-4c52-9fc0-90fa1e5ff25d None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:25:01.911023 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d2e6685-b49c-4017-b2bf-46346f753adb None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:25:01.917797 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:25:01.933668 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5196a8af-fa02-471d-90e3-bf1b2ebbdc32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:25:01.934336 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5196a8af-fa02-471d-90e3-bf1b2ebbdc32 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:25:01.934625 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5196a8af-fa02-471d-90e3-bf1b2ebbdc32 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:25:01.934905 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5196a8af-fa02-471d-90e3-bf1b2ebbdc32 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:25:01.958565 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f22ff8d4-75d8-4fc8-9457-6fd75e524c5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:25:01.959196 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22ff8d4-75d8-4fc8-9457-6fd75e524c5c None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:25:01.964802 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22ff8d4-75d8-4fc8-9457-6fd75e524c5c None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:25:01.964802 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22ff8d4-75d8-4fc8-9457-6fd75e524c5c None admin] PATH_INFO: `/v3/users` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:25:01.964802 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f22ff8d4-75d8-4fc8-9457-6fd75e524c5c None admin] RBAC: Adding query filter params (name=cinder) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:25:01.964802 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f22ff8d4-75d8-4fc8-9457-6fd75e524c5c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:25:01.964802 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f22ff8d4-75d8-4fc8-9457-6fd75e524c5c None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:25:02.006424 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 479/973] 10.222.0.33 () {56 vars in 1101 bytes} [Mon Jul 25 05:25:01 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:25:02.027264 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5196a8af-fa02-471d-90e3-bf1b2ebbdc32 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:25:02.029081 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5196a8af-fa02-471d-90e3-bf1b2ebbdc32 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:25:02.140290 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 495/974] 10.222.0.33 () {58 vars in 1282 bytes} [Mon Jul 25 05:25:01 2022] GET /identity/v3/auth/tokens => generated 1464 bytes in 398 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 25 05:25:02.145921 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f22ff8d4-75d8-4fc8-9457-6fd75e524c5c None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:25:02.249813 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:25:02.253206 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f22ff8d4-75d8-4fc8-9457-6fd75e524c5c None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:25:02.292130 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d256c58b-0976-491e-ae21-12b3e9f7741a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:25:02.294117 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d256c58b-0976-491e-ae21-12b3e9f7741a None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:25:02.294117 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d256c58b-0976-491e-ae21-12b3e9f7741a None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:25:02.294117 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d256c58b-0976-491e-ae21-12b3e9f7741a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:25:02.305030 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d256c58b-0976-491e-ae21-12b3e9f7741a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:25:02.306492 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d256c58b-0976-491e-ae21-12b3e9f7741a None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:25:02.326521 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application [None req-d256c58b-0976-491e-ae21-12b3e9f7741a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 05:25:02.326521 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:25:02.326521 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:25:02.326521 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:25:02.326521 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:25:02.326521 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:25:02.326521 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:25:02.326521 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:25:02.326521 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:25:02.326521 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:25:02.326521 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:25:02.326521 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:25:02.326521 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 05:25:02.326521 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 05:25:02.326521 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 05:25:02.327501 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 05:25:02.327501 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:25:02.327501 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:25:02.327501 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:25:02.327501 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:25:02.327501 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:25:02.327501 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:25:02.327501 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:25:02.327501 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application with Lock( Jul 25 05:25:02.327501 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:25:02.327501 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:25:02.327501 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:25:02.327501 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:25:02.327501 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:25:02.327501 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:25:02.327501 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:25:02.327501 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:25:02.328422 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 05:25:02.328422 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 05:25:02.328422 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 05:25:02.328422 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Jul 25 05:25:02.328422 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 480/975] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Jul 25 05:25:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:25:02.334894 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5196a8af-fa02-471d-90e3-bf1b2ebbdc32 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:25:02.362630 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:25:02.440523 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5196a8af-fa02-471d-90e3-bf1b2ebbdc32 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:25:02.474656 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ca7dcb-0fda-4c55-b816-1d27945470fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:25:02.475327 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ca7dcb-0fda-4c55-b816-1d27945470fb None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:25:02.475618 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ca7dcb-0fda-4c55-b816-1d27945470fb None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:25:02.475899 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ca7dcb-0fda-4c55-b816-1d27945470fb None admin] PATH_INFO: `/v3/domains` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:25:02.476625 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9ca7dcb-0fda-4c55-b816-1d27945470fb None admin] RBAC: Adding query filter params (name=Default) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:25:02.476966 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9ca7dcb-0fda-4c55-b816-1d27945470fb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:25:02.478333 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9ca7dcb-0fda-4c55-b816-1d27945470fb None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:25:02.490271 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 496/976] 10.222.0.33 () {56 vars in 1109 bytes} [Mon Jul 25 05:25:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 05:25:02.495855 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d256c58b-0976-491e-ae21-12b3e9f7741a None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:25:02.596464 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d256c58b-0976-491e-ae21-12b3e9f7741a None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:25:02.630708 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7969d2b0-978c-4eb8-a949-576129580f46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:25:02.631503 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7969d2b0-978c-4eb8-a949-576129580f46 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:25:02.631791 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7969d2b0-978c-4eb8-a949-576129580f46 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:25:02.632066 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7969d2b0-978c-4eb8-a949-576129580f46 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:25:02.639178 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7969d2b0-978c-4eb8-a949-576129580f46 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:25:02.640619 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7969d2b0-978c-4eb8-a949-576129580f46 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:25:02.654455 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application [None req-7969d2b0-978c-4eb8-a949-576129580f46 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 05:25:02.654455 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:25:02.654455 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:25:02.654455 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:25:02.654455 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:25:02.654455 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:25:02.654455 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:25:02.654455 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:25:02.654455 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:25:02.654455 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:25:02.654455 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:25:02.654455 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:25:02.654455 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 25 05:25:02.654455 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 25 05:25:02.654455 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 25 05:25:02.655393 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 25 05:25:02.655393 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:25:02.655393 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:25:02.655393 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:25:02.655393 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:25:02.655393 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:25:02.655393 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:25:02.655393 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:25:02.655393 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application with Lock( Jul 25 05:25:02.655393 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:25:02.655393 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:25:02.655393 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:25:02.655393 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:25:02.655393 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:25:02.655393 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:25:02.655393 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:25:02.655393 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:25:02.656331 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 25 05:25:02.656331 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 25 05:25:02.656331 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 25 05:25:02.656331 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 25 05:25:02.656331 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 25 05:25:02.656331 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 25 05:25:02.656331 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 05:25:02.656331 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Jul 25 05:25:02.657392 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 481/977] 10.222.0.33 () {56 vars in 1111 bytes} [Mon Jul 25 05:25:02 2022] GET /identity/v3/projects/service => generated 88 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:25:02.661483 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ca7dcb-0fda-4c55-b816-1d27945470fb None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:25:02.747791 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:25:02.766555 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ca7dcb-0fda-4c55-b816-1d27945470fb None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:25:02.772337 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912aced7-1a10-43ca-8f2e-70cea14b7899 None None] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:25:02.772628 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912aced7-1a10-43ca-8f2e-70cea14b7899 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:25:02.772940 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912aced7-1a10-43ca-8f2e-70cea14b7899 None None] PATH_INFO: `/` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:25:02.774454 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 482/978] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jul 25 05:25:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:25:02.779723 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac5ab70-7fe5-4b9b-8241-9f572cd0eafa None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:25:02.780013 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac5ab70-7fe5-4b9b-8241-9f572cd0eafa None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:25:02.780294 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac5ab70-7fe5-4b9b-8241-9f572cd0eafa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:25:02.799949 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-aac5ab70-7fe5-4b9b-8241-9f572cd0eafa None None] MFA Rules not processed for user `1eae540242d94c10b0ad33960a855f08`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:25:02.801658 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-aac5ab70-7fe5-4b9b-8241-9f572cd0eafa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:25:02.810280 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e34da513-d70c-4625-983d-17dd4ca5269c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:25:02.810442 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:25:02.810957 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34da513-d70c-4625-983d-17dd4ca5269c None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:25:02.811297 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34da513-d70c-4625-983d-17dd4ca5269c None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:25:02.811573 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34da513-d70c-4625-983d-17dd4ca5269c None admin] PATH_INFO: `/v3/projects` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:25:02.812305 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e34da513-d70c-4625-983d-17dd4ca5269c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:25:02.812629 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e34da513-d70c-4625-983d-17dd4ca5269c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:25:02.814702 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e34da513-d70c-4625-983d-17dd4ca5269c None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:25:02.827025 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 497/979] 10.222.0.33 () {56 vars in 1148 bytes} [Mon Jul 25 05:25:02 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:25:02.832819 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e34da513-d70c-4625-983d-17dd4ca5269c None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:25:02.881096 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 483/980] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jul 25 05:25:02 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 25 05:25:02.903904 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac5ab70-7fe5-4b9b-8241-9f572cd0eafa None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:25:02.945127 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e34da513-d70c-4625-983d-17dd4ca5269c None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:25:02.981795 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc3098c-1d35-4702-9d8f-daa9b1747fee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:25:02.982458 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc3098c-1d35-4702-9d8f-daa9b1747fee None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:25:02.983164 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc3098c-1d35-4702-9d8f-daa9b1747fee None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:25:02.983446 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc3098c-1d35-4702-9d8f-daa9b1747fee None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:25:02.984292 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efc3098c-1d35-4702-9d8f-daa9b1747fee None admin] RBAC: Adding query filter params (role.id=e85cbdc7a77a4fc29d71c8a3ae0f56f4, scope.project.id=92afba266cce4d1bad4134c351e74be3, user.id=3352531be1b54f76b864383ccc2e5ec2) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:25:02.984988 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efc3098c-1d35-4702-9d8f-daa9b1747fee None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:25:02.986770 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efc3098c-1d35-4702-9d8f-daa9b1747fee None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:25:03.000825 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 498/981] 10.222.0.33 () {56 vars in 1374 bytes} [Mon Jul 25 05:25:02 2022] GET /identity/v3/role_assignments?user.id=3352531be1b54f76b864383ccc2e5ec2&scope.project.id=92afba266cce4d1bad4134c351e74be3&role.id=e85cbdc7a77a4fc29d71c8a3ae0f56f4 => generated 608 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 05:25:03.019249 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac5ab70-7fe5-4b9b-8241-9f572cd0eafa None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:25:03.058270 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d51ca04-991b-4f98-b7ad-42456724eca7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:25:03.058973 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d51ca04-991b-4f98-b7ad-42456724eca7 None admin] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:25:03.059256 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d51ca04-991b-4f98-b7ad-42456724eca7 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:25:03.059530 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d51ca04-991b-4f98-b7ad-42456724eca7 None admin] PATH_INFO: `/v3/services` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:25:03.060274 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d51ca04-991b-4f98-b7ad-42456724eca7 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:25:03.061718 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d51ca04-991b-4f98-b7ad-42456724eca7 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:25:03.077798 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 484/982] 10.222.0.33 () {60 vars in 1141 bytes} [Mon Jul 25 05:25:02 2022] POST /identity/v3/services => generated 242 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 25 05:25:03.315823 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807642be-ef56-4f51-9795-4319ae33ce4a None None] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:25:03.316379 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807642be-ef56-4f51-9795-4319ae33ce4a None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:25:03.316379 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807642be-ef56-4f51-9795-4319ae33ce4a None None] PATH_INFO: `/` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:25:03.317851 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 499/983] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jul 25 05:25:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:25:03.323280 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32550e92-515e-4865-8b56-5e9d58533f0d None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:25:03.323632 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32550e92-515e-4865-8b56-5e9d58533f0d None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:25:03.323945 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32550e92-515e-4865-8b56-5e9d58533f0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:25:03.328978 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:25:03.343202 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-32550e92-515e-4865-8b56-5e9d58533f0d None None] MFA Rules not processed for user `1eae540242d94c10b0ad33960a855f08`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:25:03.344428 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-32550e92-515e-4865-8b56-5e9d58533f0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:25:03.421423 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 485/984] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jul 25 05:25:03 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 25 05:25:03.444796 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-807642be-ef56-4f51-9795-4319ae33ce4a None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:25:03.453929 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:25:03.547766 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-807642be-ef56-4f51-9795-4319ae33ce4a None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:25:03.584182 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef018fb-976c-4245-93c5-c9fab43595cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:25:03.584938 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef018fb-976c-4245-93c5-c9fab43595cf None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:25:03.585229 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef018fb-976c-4245-93c5-c9fab43595cf None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:25:03.585509 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef018fb-976c-4245-93c5-c9fab43595cf None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:25:03.586165 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ef018fb-976c-4245-93c5-c9fab43595cf None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:25:03.587543 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ef018fb-976c-4245-93c5-c9fab43595cf None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:25:03.599803 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application [None req-8ef018fb-976c-4245-93c5-c9fab43595cf None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 25 05:25:03.599803 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:25:03.599803 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 25 05:25:03.599803 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 25 05:25:03.599803 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 25 05:25:03.599803 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 25 05:25:03.599803 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 25 05:25:03.599803 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 25 05:25:03.599803 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 25 05:25:03.599803 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Jul 25 05:25:03.599803 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 25 05:25:03.599803 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Jul 25 05:25:03.599803 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:25:03.599803 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:25:03.599803 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:25:03.599803 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:25:03.599803 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:25:03.600909 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:25:03.600909 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:25:03.600909 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:25:03.600909 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:25:03.600909 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:25:03.600909 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:25:03.600909 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 25 05:25:03.600909 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 25 05:25:03.600909 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 25 05:25:03.600909 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 25 05:25:03.600909 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:25:03.600909 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:25:03.600909 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:25:03.600909 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:25:03.600909 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:25:03.600909 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:25:03.601784 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:25:03.601784 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application with Lock( Jul 25 05:25:03.601784 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:25:03.601784 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:25:03.601784 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:25:03.601784 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:25:03.601784 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:25:03.601784 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:25:03.601784 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:25:03.601784 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:25:03.601784 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 25 05:25:03.601784 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 25 05:25:03.601784 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 25 05:25:03.601784 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Jul 25 05:25:03.603090 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 500/985] 10.222.0.33 () {56 vars in 1126 bytes} [Mon Jul 25 05:25:03 2022] GET /identity/v3/services/object-store => generated 93 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:25:03.607244 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32550e92-515e-4865-8b56-5e9d58533f0d None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:25:03.706633 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32550e92-515e-4865-8b56-5e9d58533f0d None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:25:03.732575 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:25:03.741646 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e1365b-e9a6-4bf3-96a3-ebdce5b3f91d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:25:03.742312 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e1365b-e9a6-4bf3-96a3-ebdce5b3f91d None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:25:03.742603 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e1365b-e9a6-4bf3-96a3-ebdce5b3f91d None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:25:03.742879 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e1365b-e9a6-4bf3-96a3-ebdce5b3f91d None admin] PATH_INFO: `/v3/services` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:25:03.743590 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46e1365b-e9a6-4bf3-96a3-ebdce5b3f91d None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:25:03.743898 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46e1365b-e9a6-4bf3-96a3-ebdce5b3f91d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:25:03.745826 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46e1365b-e9a6-4bf3-96a3-ebdce5b3f91d None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:25:03.755502 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 486/986] 10.222.0.33 () {56 vars in 1122 bytes} [Mon Jul 25 05:25:03 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 05:25:03.756615 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef018fb-976c-4245-93c5-c9fab43595cf None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:25:03.834631 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:25:03.867488 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef018fb-976c-4245-93c5-c9fab43595cf None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:25:03.901018 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec66909-6603-4459-a21f-ce4acbd26476 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:25:03.901673 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec66909-6603-4459-a21f-ce4acbd26476 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:25:03.901957 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec66909-6603-4459-a21f-ce4acbd26476 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:25:03.902232 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec66909-6603-4459-a21f-ce4acbd26476 None admin] PATH_INFO: `/v3/services` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:25:03.902934 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ec66909-6603-4459-a21f-ce4acbd26476 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:25:03.903241 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ec66909-6603-4459-a21f-ce4acbd26476 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:25:03.904590 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ec66909-6603-4459-a21f-ce4acbd26476 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:25:03.914180 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 501/987] 10.222.0.33 () {56 vars in 1122 bytes} [Mon Jul 25 05:25:03 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:25:03.915439 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e1365b-e9a6-4bf3-96a3-ebdce5b3f91d None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:25:04.018784 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e1365b-e9a6-4bf3-96a3-ebdce5b3f91d None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:25:04.052510 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e0795f-0e2b-4204-bbf4-4b8bfbaf34c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:25:04.053297 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e0795f-0e2b-4204-bbf4-4b8bfbaf34c3 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:25:04.053482 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e0795f-0e2b-4204-bbf4-4b8bfbaf34c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:25:04.053749 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e0795f-0e2b-4204-bbf4-4b8bfbaf34c3 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:25:04.054553 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6e0795f-0e2b-4204-bbf4-4b8bfbaf34c3 None admin] RBAC: Adding query filter params (interface=admin, service_id=f5c4b2feadf540eeb517ed0d942d220c, region_id=RegionOne) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:25:04.054867 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6e0795f-0e2b-4204-bbf4-4b8bfbaf34c3 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:25:04.056218 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6e0795f-0e2b-4204-bbf4-4b8bfbaf34c3 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:25:04.062237 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 487/988] 10.222.0.33 () {56 vars in 1249 bytes} [Mon Jul 25 05:25:03 2022] GET /identity/v3/endpoints?service_id=f5c4b2feadf540eeb517ed0d942d220c&interface=admin®ion_id=RegionOne => generated 193 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:25:04.067811 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec66909-6603-4459-a21f-ce4acbd26476 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:25:04.171842 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec66909-6603-4459-a21f-ce4acbd26476 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:25:04.205992 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60cf8353-8a38-46fd-9029-834616375c2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:25:04.206661 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60cf8353-8a38-46fd-9029-834616375c2c None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:25:04.206951 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60cf8353-8a38-46fd-9029-834616375c2c None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:25:04.207233 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60cf8353-8a38-46fd-9029-834616375c2c None admin] PATH_INFO: `/v3/services` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:25:04.207926 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60cf8353-8a38-46fd-9029-834616375c2c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:25:04.209323 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60cf8353-8a38-46fd-9029-834616375c2c None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:25:04.210408 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:25:04.223872 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 502/989] 10.222.0.33 () {56 vars in 1087 bytes} [Mon Jul 25 05:25:04 2022] GET /identity/v3/services => generated 1727 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 05:25:05.081986 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01b308b-005a-4f56-a95b-908dd009ee9e None None] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:25:05.082497 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01b308b-005a-4f56-a95b-908dd009ee9e None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:25:05.082497 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01b308b-005a-4f56-a95b-908dd009ee9e None None] PATH_INFO: `/` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:25:05.083977 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 488/990] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jul 25 05:25:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:25:05.089532 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f772b531-9c9d-4475-b529-af74475a7969 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:25:05.089984 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f772b531-9c9d-4475-b529-af74475a7969 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:25:05.090140 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f772b531-9c9d-4475-b529-af74475a7969 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:25:05.107379 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-f772b531-9c9d-4475-b529-af74475a7969 None None] MFA Rules not processed for user `1eae540242d94c10b0ad33960a855f08`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:25:05.108582 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-f772b531-9c9d-4475-b529-af74475a7969 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:25:05.152128 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a23fd5-6b95-4e9c-aed0-8e9c8a03dd0b None None] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:25:05.152509 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a23fd5-6b95-4e9c-aed0-8e9c8a03dd0b None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:25:05.153049 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a23fd5-6b95-4e9c-aed0-8e9c8a03dd0b None None] PATH_INFO: `/` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:25:05.154679 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 489/991] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jul 25 05:25:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:25:05.159608 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1f4d70-d1b3-4857-8a4f-9bf506935cd4 None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:25:05.160054 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1f4d70-d1b3-4857-8a4f-9bf506935cd4 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:25:05.160499 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1f4d70-d1b3-4857-8a4f-9bf506935cd4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:25:05.177935 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 503/992] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jul 25 05:25:05 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 25 05:25:05.178105 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-9f1f4d70-d1b3-4857-8a4f-9bf506935cd4 None None] MFA Rules not processed for user `1eae540242d94c10b0ad33960a855f08`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:25:05.179311 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-9f1f4d70-d1b3-4857-8a4f-9bf506935cd4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:25:05.199883 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f772b531-9c9d-4475-b529-af74475a7969 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:25:05.236764 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:25:05.254219 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 490/993] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jul 25 05:25:05 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 25 05:25:05.276693 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f1f4d70-d1b3-4857-8a4f-9bf506935cd4 None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:25:05.307461 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f772b531-9c9d-4475-b529-af74475a7969 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:25:05.335214 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:25:05.344640 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6656a03-a182-4992-baac-3ee4ea4ffc8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:25:05.345474 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6656a03-a182-4992-baac-3ee4ea4ffc8f None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:25:05.345760 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6656a03-a182-4992-baac-3ee4ea4ffc8f None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:25:05.346038 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6656a03-a182-4992-baac-3ee4ea4ffc8f None admin] PATH_INFO: `/v3/services/image` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:25:05.346691 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6656a03-a182-4992-baac-3ee4ea4ffc8f None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:25:05.348051 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6656a03-a182-4992-baac-3ee4ea4ffc8f None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:25:05.360079 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application [None req-f6656a03-a182-4992-baac-3ee4ea4ffc8f None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jul 25 05:25:05.360079 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:25:05.360079 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 25 05:25:05.360079 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 25 05:25:05.360079 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 25 05:25:05.360079 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 25 05:25:05.360079 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 25 05:25:05.360079 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 25 05:25:05.360079 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 25 05:25:05.360079 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Jul 25 05:25:05.360079 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 25 05:25:05.360079 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Jul 25 05:25:05.360079 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:25:05.360079 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:25:05.360079 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:25:05.360079 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:25:05.360079 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:25:05.361157 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:25:05.361157 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:25:05.361157 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:25:05.361157 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:25:05.361157 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:25:05.361157 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:25:05.361157 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 25 05:25:05.361157 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 25 05:25:05.361157 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 25 05:25:05.361157 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 25 05:25:05.361157 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:25:05.361157 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:25:05.361157 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:25:05.361157 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:25:05.361157 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:25:05.361157 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:25:05.362044 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:25:05.362044 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application with Lock( Jul 25 05:25:05.362044 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:25:05.362044 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:25:05.362044 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:25:05.362044 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:25:05.362044 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:25:05.362044 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:25:05.362044 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:25:05.362044 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:25:05.362044 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 25 05:25:05.362044 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 25 05:25:05.362044 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 25 05:25:05.362044 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Jul 25 05:25:05.362044 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 504/994] 10.222.0.33 () {56 vars in 1105 bytes} [Mon Jul 25 05:25:05 2022] GET /identity/v3/services/image => generated 86 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 25 05:25:05.367099 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6656a03-a182-4992-baac-3ee4ea4ffc8f None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:25:05.395099 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f1f4d70-d1b3-4857-8a4f-9bf506935cd4 None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:25:05.432541 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b28d14e-c7cd-42e0-b60c-8e6ef062bc55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:25:05.433360 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b28d14e-c7cd-42e0-b60c-8e6ef062bc55 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:25:05.433786 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b28d14e-c7cd-42e0-b60c-8e6ef062bc55 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:25:05.434078 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b28d14e-c7cd-42e0-b60c-8e6ef062bc55 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:25:05.435199 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b28d14e-c7cd-42e0-b60c-8e6ef062bc55 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:25:05.436569 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b28d14e-c7cd-42e0-b60c-8e6ef062bc55 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:25:05.449649 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application [None req-1b28d14e-c7cd-42e0-b60c-8e6ef062bc55 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jul 25 05:25:05.449649 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:25:05.449649 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 25 05:25:05.449649 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 25 05:25:05.449649 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 25 05:25:05.449649 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 25 05:25:05.449649 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 25 05:25:05.449649 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 25 05:25:05.449649 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 25 05:25:05.449649 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Jul 25 05:25:05.449649 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 25 05:25:05.449649 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Jul 25 05:25:05.449649 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:25:05.449649 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:25:05.449649 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:25:05.449649 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:25:05.449649 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:25:05.450670 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:25:05.450670 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:25:05.450670 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:25:05.450670 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:25:05.450670 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:25:05.450670 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:25:05.450670 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 25 05:25:05.450670 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 25 05:25:05.450670 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 25 05:25:05.450670 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 25 05:25:05.450670 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:25:05.450670 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:25:05.450670 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:25:05.450670 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:25:05.450670 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:25:05.450670 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:25:05.451528 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:25:05.451528 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application with Lock( Jul 25 05:25:05.451528 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:25:05.451528 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:25:05.451528 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:25:05.451528 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:25:05.451528 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:25:05.451528 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:25:05.451528 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:25:05.451528 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:25:05.451528 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 25 05:25:05.451528 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 25 05:25:05.451528 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 25 05:25:05.451528 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Jul 25 05:25:05.451528 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 491/995] 10.222.0.33 () {56 vars in 1111 bytes} [Mon Jul 25 05:25:05 2022] GET /identity/v3/services/compute => generated 88 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 25 05:25:05.456726 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b28d14e-c7cd-42e0-b60c-8e6ef062bc55 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:25:05.471972 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6656a03-a182-4992-baac-3ee4ea4ffc8f None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:25:05.506406 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ca9908-4be6-4983-b657-3fc741716aa5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:25:05.507060 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ca9908-4be6-4983-b657-3fc741716aa5 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:25:05.507344 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ca9908-4be6-4983-b657-3fc741716aa5 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:25:05.507618 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ca9908-4be6-4983-b657-3fc741716aa5 None admin] PATH_INFO: `/v3/services` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:25:05.508317 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4ca9908-4be6-4983-b657-3fc741716aa5 None admin] RBAC: Adding query filter params (name=image) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:25:05.508626 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4ca9908-4be6-4983-b657-3fc741716aa5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:25:05.510003 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4ca9908-4be6-4983-b657-3fc741716aa5 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:25:05.514979 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 505/996] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Jul 25 05:25:05 2022] GET /identity/v3/services?name=image => generated 122 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 05:25:05.518187 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:25:05.521083 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ca9908-4be6-4983-b657-3fc741716aa5 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:25:05.578777 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:25:05.582068 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b28d14e-c7cd-42e0-b60c-8e6ef062bc55 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:25:05.617592 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee3a107-b712-4eeb-ad79-8cfb823f9023 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:25:05.618209 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee3a107-b712-4eeb-ad79-8cfb823f9023 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:25:05.618499 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee3a107-b712-4eeb-ad79-8cfb823f9023 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:25:05.618780 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee3a107-b712-4eeb-ad79-8cfb823f9023 None admin] PATH_INFO: `/v3/services` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:25:05.619616 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aee3a107-b712-4eeb-ad79-8cfb823f9023 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:25:05.620612 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aee3a107-b712-4eeb-ad79-8cfb823f9023 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:25:05.622169 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aee3a107-b712-4eeb-ad79-8cfb823f9023 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:25:05.629203 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ca9908-4be6-4983-b657-3fc741716aa5 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:25:05.632467 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 492/997] 10.222.0.33 () {56 vars in 1112 bytes} [Mon Jul 25 05:25:05 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 05:25:05.634014 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee3a107-b712-4eeb-ad79-8cfb823f9023 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:25:05.668588 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aea739e-e959-432e-926f-0f43f9d8f4f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:25:05.670423 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aea739e-e959-432e-926f-0f43f9d8f4f6 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:25:05.670902 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aea739e-e959-432e-926f-0f43f9d8f4f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:25:05.671380 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aea739e-e959-432e-926f-0f43f9d8f4f6 None admin] PATH_INFO: `/v3/services` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:25:05.672265 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aea739e-e959-432e-926f-0f43f9d8f4f6 None admin] RBAC: Adding query filter params (type=image) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:25:05.672772 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aea739e-e959-432e-926f-0f43f9d8f4f6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:25:05.674759 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aea739e-e959-432e-926f-0f43f9d8f4f6 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:25:05.680155 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 506/998] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Jul 25 05:25:05 2022] GET /identity/v3/services?type=image => generated 122 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 05:25:05.742321 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee3a107-b712-4eeb-ad79-8cfb823f9023 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:25:05.758799 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:25:05.782720 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a91c4e-0c5a-4754-8bbd-aaa498f34f09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:25:05.783409 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a91c4e-0c5a-4754-8bbd-aaa498f34f09 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:25:05.783698 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a91c4e-0c5a-4754-8bbd-aaa498f34f09 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:25:05.783978 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a91c4e-0c5a-4754-8bbd-aaa498f34f09 None admin] PATH_INFO: `/v3/services` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:25:05.784707 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4a91c4e-0c5a-4754-8bbd-aaa498f34f09 None admin] RBAC: Adding query filter params (type=compute) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:25:05.785069 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4a91c4e-0c5a-4754-8bbd-aaa498f34f09 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:25:05.786463 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4a91c4e-0c5a-4754-8bbd-aaa498f34f09 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:25:05.793126 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 493/999] 10.222.0.33 () {56 vars in 1112 bytes} [Mon Jul 25 05:25:05 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 05:25:05.800166 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aea739e-e959-432e-926f-0f43f9d8f4f6 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:25:05.911834 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aea739e-e959-432e-926f-0f43f9d8f4f6 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:25:05.933450 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:25:05.948969 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d197efd4-8e82-49ac-ac3a-6d8b33b0ca5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:25:05.949632 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d197efd4-8e82-49ac-ac3a-6d8b33b0ca5f None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:25:05.949917 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d197efd4-8e82-49ac-ac3a-6d8b33b0ca5f None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:25:05.950191 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d197efd4-8e82-49ac-ac3a-6d8b33b0ca5f None admin] PATH_INFO: `/v3/endpoints` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:25:05.950985 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d197efd4-8e82-49ac-ac3a-6d8b33b0ca5f None admin] RBAC: Adding query filter params (interface=public, service_id=c799a77c00864bc4be1f318bd5752788, region_id=RegionOne) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:25:05.951295 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d197efd4-8e82-49ac-ac3a-6d8b33b0ca5f None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:25:05.952655 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d197efd4-8e82-49ac-ac3a-6d8b33b0ca5f None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:25:05.958380 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 507/1000] 10.222.0.33 () {56 vars in 1251 bytes} [Mon Jul 25 05:25:05 2022] GET /identity/v3/endpoints?service_id=c799a77c00864bc4be1f318bd5752788&interface=public®ion_id=RegionOne => generated 194 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:25:05.964825 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a91c4e-0c5a-4754-8bbd-aaa498f34f09 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:25:06.066270 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a91c4e-0c5a-4754-8bbd-aaa498f34f09 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:25:06.099594 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a033d7c3-b362-4c9a-a99f-2274083dc907 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:25:06.100241 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a033d7c3-b362-4c9a-a99f-2274083dc907 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:25:06.100528 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a033d7c3-b362-4c9a-a99f-2274083dc907 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:25:06.100817 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a033d7c3-b362-4c9a-a99f-2274083dc907 None admin] PATH_INFO: `/v3/services` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:25:06.101504 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a033d7c3-b362-4c9a-a99f-2274083dc907 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:25:06.102858 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a033d7c3-b362-4c9a-a99f-2274083dc907 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:25:06.116865 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 494/1001] 10.222.0.33 () {56 vars in 1087 bytes} [Mon Jul 25 05:25:05 2022] GET /identity/v3/services => generated 1727 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 25 05:25:06.274549 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aece4fec-e921-42f1-b0ea-e4deb7eb13b4 None None] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:25:06.275066 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aece4fec-e921-42f1-b0ea-e4deb7eb13b4 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:25:06.275066 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aece4fec-e921-42f1-b0ea-e4deb7eb13b4 None None] PATH_INFO: `/` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:25:06.277577 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 508/1002] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jul 25 05:25:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:25:06.282884 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0518667-361a-499a-9dfc-9e043dd1a400 None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:25:06.283173 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0518667-361a-499a-9dfc-9e043dd1a400 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:25:06.283452 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0518667-361a-499a-9dfc-9e043dd1a400 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:25:06.300521 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-c0518667-361a-499a-9dfc-9e043dd1a400 None None] MFA Rules not processed for user `1eae540242d94c10b0ad33960a855f08`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:25:06.301800 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-c0518667-361a-499a-9dfc-9e043dd1a400 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:25:06.339327 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:25:06.372062 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 495/1003] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jul 25 05:25:06 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 25 05:25:06.395388 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aece4fec-e921-42f1-b0ea-e4deb7eb13b4 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:25:06.498048 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aece4fec-e921-42f1-b0ea-e4deb7eb13b4 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:25:06.531351 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f1e4a6-d0ff-4571-bfdd-c7821e95adaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:25:06.532085 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f1e4a6-d0ff-4571-bfdd-c7821e95adaa None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:25:06.532376 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f1e4a6-d0ff-4571-bfdd-c7821e95adaa None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:25:06.532657 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f1e4a6-d0ff-4571-bfdd-c7821e95adaa None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:25:06.533337 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1f1e4a6-d0ff-4571-bfdd-c7821e95adaa None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:25:06.534699 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1f1e4a6-d0ff-4571-bfdd-c7821e95adaa None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:25:06.546907 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application [None req-b1f1e4a6-d0ff-4571-bfdd-c7821e95adaa None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 25 05:25:06.546907 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:25:06.546907 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 25 05:25:06.546907 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 25 05:25:06.546907 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 25 05:25:06.546907 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 25 05:25:06.546907 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 25 05:25:06.546907 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 25 05:25:06.546907 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 25 05:25:06.546907 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Jul 25 05:25:06.546907 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 25 05:25:06.546907 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Jul 25 05:25:06.546907 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:25:06.546907 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:25:06.546907 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:25:06.546907 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:25:06.546907 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:25:06.548015 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:25:06.548015 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:25:06.548015 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:25:06.548015 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:25:06.548015 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:25:06.548015 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:25:06.548015 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 25 05:25:06.548015 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 25 05:25:06.548015 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 25 05:25:06.548015 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 25 05:25:06.548015 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:25:06.548015 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:25:06.548015 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:25:06.548015 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:25:06.548015 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:25:06.548015 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:25:06.549661 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:25:06.549661 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application with Lock( Jul 25 05:25:06.549661 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:25:06.549661 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:25:06.549661 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:25:06.549661 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:25:06.549661 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:25:06.549661 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:25:06.549661 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:25:06.549661 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:25:06.549661 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 25 05:25:06.549661 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 25 05:25:06.549661 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 25 05:25:06.549661 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Jul 25 05:25:06.553343 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 509/1004] 10.222.0.33 () {56 vars in 1126 bytes} [Mon Jul 25 05:25:06 2022] GET /identity/v3/services/object-store => generated 93 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:25:06.554324 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0518667-361a-499a-9dfc-9e043dd1a400 None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:25:06.656060 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0518667-361a-499a-9dfc-9e043dd1a400 None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:25:06.690747 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f01f80-30c5-44db-933e-51118685dde3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:25:06.691415 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f01f80-30c5-44db-933e-51118685dde3 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:25:06.691701 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f01f80-30c5-44db-933e-51118685dde3 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:25:06.691977 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f01f80-30c5-44db-933e-51118685dde3 None admin] PATH_INFO: `/v3/services` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:25:06.692688 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04f01f80-30c5-44db-933e-51118685dde3 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:25:06.693029 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04f01f80-30c5-44db-933e-51118685dde3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:25:06.694390 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04f01f80-30c5-44db-933e-51118685dde3 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:25:06.704088 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 496/1005] 10.222.0.33 () {56 vars in 1122 bytes} [Mon Jul 25 05:25:06 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:25:06.705626 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f1e4a6-d0ff-4571-bfdd-c7821e95adaa None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:25:06.756300 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:25:06.818131 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f1e4a6-d0ff-4571-bfdd-c7821e95adaa None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:25:06.852832 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed728b7-2d15-4480-a8f7-9f16e88bcfdc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:25:06.853508 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed728b7-2d15-4480-a8f7-9f16e88bcfdc None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:25:06.853795 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed728b7-2d15-4480-a8f7-9f16e88bcfdc None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:25:06.854862 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed728b7-2d15-4480-a8f7-9f16e88bcfdc None admin] PATH_INFO: `/v3/services` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:25:06.855886 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ed728b7-2d15-4480-a8f7-9f16e88bcfdc None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:25:06.856201 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ed728b7-2d15-4480-a8f7-9f16e88bcfdc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:25:06.857599 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ed728b7-2d15-4480-a8f7-9f16e88bcfdc None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:25:06.867178 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 510/1006] 10.222.0.33 () {56 vars in 1122 bytes} [Mon Jul 25 05:25:06 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 05:25:06.868821 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f01f80-30c5-44db-933e-51118685dde3 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:25:06.905764 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:25:06.973578 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f01f80-30c5-44db-933e-51118685dde3 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:25:07.005969 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda42daf-419f-46a1-949a-d04a12593cd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:25:07.006616 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda42daf-419f-46a1-949a-d04a12593cd2 None admin] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:25:07.006897 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda42daf-419f-46a1-949a-d04a12593cd2 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:25:07.007166 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda42daf-419f-46a1-949a-d04a12593cd2 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:25:07.007897 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dda42daf-419f-46a1-949a-d04a12593cd2 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:25:07.009280 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dda42daf-419f-46a1-949a-d04a12593cd2 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:25:07.032808 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 497/1007] 10.222.0.33 () {60 vars in 1144 bytes} [Mon Jul 25 05:25:06 2022] POST /identity/v3/endpoints => generated 325 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 25 05:25:07.732006 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b5e246-a6bd-48f1-89a5-aa29c069dd74 None None] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:25:07.732564 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b5e246-a6bd-48f1-89a5-aa29c069dd74 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:25:07.732564 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b5e246-a6bd-48f1-89a5-aa29c069dd74 None None] PATH_INFO: `/` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:25:07.734165 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 511/1008] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jul 25 05:25:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:25:07.739201 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fce03a6-f28c-428d-9da0-6d65d778a3e1 None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:25:07.739651 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fce03a6-f28c-428d-9da0-6d65d778a3e1 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:25:07.739801 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fce03a6-f28c-428d-9da0-6d65d778a3e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:25:07.757494 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-4fce03a6-f28c-428d-9da0-6d65d778a3e1 None None] MFA Rules not processed for user `1eae540242d94c10b0ad33960a855f08`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:25:07.758700 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-4fce03a6-f28c-428d-9da0-6d65d778a3e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:25:07.836432 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 498/1009] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jul 25 05:25:07 2022] POST /identity/v3/auth/tokens => generated 1707 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 25 05:25:07.861168 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b5e246-a6bd-48f1-89a5-aa29c069dd74 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:25:07.967764 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b5e246-a6bd-48f1-89a5-aa29c069dd74 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:25:07.981913 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:25:08.004327 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-799d4404-7822-41e6-8477-f3287034e79b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:25:08.005248 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799d4404-7822-41e6-8477-f3287034e79b None admin] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:25:08.005686 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799d4404-7822-41e6-8477-f3287034e79b None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:25:08.006116 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799d4404-7822-41e6-8477-f3287034e79b None admin] PATH_INFO: `/v3/services` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:25:08.007023 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-799d4404-7822-41e6-8477-f3287034e79b None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:25:08.008542 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-799d4404-7822-41e6-8477-f3287034e79b None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:25:08.023011 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 512/1010] 10.222.0.33 () {60 vars in 1141 bytes} [Mon Jul 25 05:25:07 2022] POST /identity/v3/services => generated 242 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 05:25:08.186653 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75dde307-790b-4347-8856-0800f6d96ff0 None None] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:25:08.186944 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75dde307-790b-4347-8856-0800f6d96ff0 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:25:08.187231 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75dde307-790b-4347-8856-0800f6d96ff0 None None] PATH_INFO: `/` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:25:08.188759 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 499/1011] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jul 25 05:25:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:25:08.194112 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05df89af-39d3-4700-b461-91c3ba7b5822 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:25:08.194404 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05df89af-39d3-4700-b461-91c3ba7b5822 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:25:08.194684 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05df89af-39d3-4700-b461-91c3ba7b5822 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:25:08.211714 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-05df89af-39d3-4700-b461-91c3ba7b5822 None None] MFA Rules not processed for user `1eae540242d94c10b0ad33960a855f08`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:25:08.213234 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-05df89af-39d3-4700-b461-91c3ba7b5822 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:25:08.285336 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 513/1012] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jul 25 05:25:08 2022] POST /identity/v3/auth/tokens => generated 1803 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 25 05:25:08.308364 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75dde307-790b-4347-8856-0800f6d96ff0 None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:25:08.332591 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:25:08.410743 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75dde307-790b-4347-8856-0800f6d96ff0 None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:25:08.443684 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a3b30b3-c761-447b-90a5-138c75781715 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:25:08.444476 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3b30b3-c761-447b-90a5-138c75781715 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:25:08.444758 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3b30b3-c761-447b-90a5-138c75781715 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:25:08.445123 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3b30b3-c761-447b-90a5-138c75781715 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:25:08.445709 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a3b30b3-c761-447b-90a5-138c75781715 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:25:08.447074 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a3b30b3-c761-447b-90a5-138c75781715 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:25:08.459057 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application [None req-8a3b30b3-c761-447b-90a5-138c75781715 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jul 25 05:25:08.459057 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:25:08.459057 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 25 05:25:08.459057 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 25 05:25:08.459057 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 25 05:25:08.459057 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 25 05:25:08.459057 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 25 05:25:08.459057 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 25 05:25:08.459057 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 25 05:25:08.459057 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Jul 25 05:25:08.459057 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 25 05:25:08.459057 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Jul 25 05:25:08.459057 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:25:08.459057 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:25:08.459057 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:25:08.459057 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:25:08.459057 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:25:08.460074 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:25:08.460074 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:25:08.460074 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:25:08.460074 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:25:08.460074 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:25:08.460074 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:25:08.460074 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 25 05:25:08.460074 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 25 05:25:08.460074 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 25 05:25:08.460074 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 25 05:25:08.460074 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:25:08.460074 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:25:08.460074 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:25:08.460074 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:25:08.460074 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:25:08.460074 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:25:08.460953 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:25:08.460953 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application with Lock( Jul 25 05:25:08.460953 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:25:08.460953 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:25:08.460953 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:25:08.460953 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:25:08.460953 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:25:08.460953 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:25:08.460953 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:25:08.460953 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:25:08.460953 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 25 05:25:08.460953 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 25 05:25:08.460953 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 25 05:25:08.460953 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Jul 25 05:25:08.464861 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 500/1013] 10.222.0.33 () {56 vars in 1111 bytes} [Mon Jul 25 05:25:08 2022] GET /identity/v3/services/compute => generated 88 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:25:08.466491 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05df89af-39d3-4700-b461-91c3ba7b5822 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:25:08.563587 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:25:08.568604 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05df89af-39d3-4700-b461-91c3ba7b5822 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:25:08.603320 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e9859e9-90f5-49a7-9d22-360b3835931b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:25:08.603998 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9859e9-90f5-49a7-9d22-360b3835931b None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:25:08.604989 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9859e9-90f5-49a7-9d22-360b3835931b None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:25:08.605279 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9859e9-90f5-49a7-9d22-360b3835931b None admin] PATH_INFO: `/v3/services` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:25:08.605998 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e9859e9-90f5-49a7-9d22-360b3835931b None admin] RBAC: Adding query filter params (name=compute) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:25:08.606306 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e9859e9-90f5-49a7-9d22-360b3835931b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:25:08.607683 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e9859e9-90f5-49a7-9d22-360b3835931b None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:25:08.612895 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 514/1014] 10.222.0.33 () {56 vars in 1112 bytes} [Mon Jul 25 05:25:08 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:25:08.618297 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a3b30b3-c761-447b-90a5-138c75781715 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:25:08.703851 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:25:08.719956 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a3b30b3-c761-447b-90a5-138c75781715 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:25:08.753645 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b22c49d-0453-46e8-8c3a-e4bb48e71bc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:25:08.754321 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b22c49d-0453-46e8-8c3a-e4bb48e71bc4 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:25:08.754573 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b22c49d-0453-46e8-8c3a-e4bb48e71bc4 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:25:08.754846 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b22c49d-0453-46e8-8c3a-e4bb48e71bc4 None admin] PATH_INFO: `/v3/services` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:25:08.755549 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b22c49d-0453-46e8-8c3a-e4bb48e71bc4 None admin] RBAC: Adding query filter params (type=compute) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:25:08.755855 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b22c49d-0453-46e8-8c3a-e4bb48e71bc4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:25:08.757245 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b22c49d-0453-46e8-8c3a-e4bb48e71bc4 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:25:08.766842 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 501/1015] 10.222.0.33 () {56 vars in 1112 bytes} [Mon Jul 25 05:25:08 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:25:08.768028 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e9859e9-90f5-49a7-9d22-360b3835931b None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:25:08.877270 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e9859e9-90f5-49a7-9d22-360b3835931b None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:25:08.911278 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e17314-2080-43e7-b772-2fdbbc7afe1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:25:08.911927 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e17314-2080-43e7-b772-2fdbbc7afe1a None admin] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:25:08.912209 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e17314-2080-43e7-b772-2fdbbc7afe1a None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:25:08.912483 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e17314-2080-43e7-b772-2fdbbc7afe1a None admin] PATH_INFO: `/v3/endpoints` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:25:08.913248 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61e17314-2080-43e7-b772-2fdbbc7afe1a None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:25:08.914604 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61e17314-2080-43e7-b772-2fdbbc7afe1a None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:25:08.934755 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 515/1016] 10.222.0.33 () {60 vars in 1144 bytes} [Mon Jul 25 05:25:08 2022] POST /identity/v3/endpoints => generated 334 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 05:25:09.108846 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b5d39b-6b41-4ebc-bd9d-2ffa36915427 None None] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:25:09.109165 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b5d39b-6b41-4ebc-bd9d-2ffa36915427 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:25:09.109416 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b5d39b-6b41-4ebc-bd9d-2ffa36915427 None None] PATH_INFO: `/` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:25:09.110936 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 502/1017] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jul 25 05:25:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:25:09.116128 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f7341e-219c-4410-a76a-83fb8668f0d7 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:25:09.116421 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f7341e-219c-4410-a76a-83fb8668f0d7 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:25:09.116697 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f7341e-219c-4410-a76a-83fb8668f0d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:25:09.134721 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-30f7341e-219c-4410-a76a-83fb8668f0d7 None None] MFA Rules not processed for user `1eae540242d94c10b0ad33960a855f08`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:25:09.135853 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-30f7341e-219c-4410-a76a-83fb8668f0d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:25:09.150380 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:25:09.219497 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 516/1018] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jul 25 05:25:09 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 25 05:25:09.249167 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b5d39b-6b41-4ebc-bd9d-2ffa36915427 None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:25:09.394333 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b5d39b-6b41-4ebc-bd9d-2ffa36915427 None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:25:09.431464 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea62a1bc-c785-4d93-8a4c-c79726dba671 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:25:09.434108 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea62a1bc-c785-4d93-8a4c-c79726dba671 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:25:09.434108 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea62a1bc-c785-4d93-8a4c-c79726dba671 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:25:09.434108 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea62a1bc-c785-4d93-8a4c-c79726dba671 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:25:09.436945 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea62a1bc-c785-4d93-8a4c-c79726dba671 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:25:09.436945 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea62a1bc-c785-4d93-8a4c-c79726dba671 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:25:09.443698 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 503/1019] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Jul 25 05:25:09 2022] GET /identity/v3/domains/default => generated 199 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:25:09.444801 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f7341e-219c-4410-a76a-83fb8668f0d7 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:25:09.582119 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f7341e-219c-4410-a76a-83fb8668f0d7 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:25:09.625279 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efad5732-290b-40db-94c8-e068ce1d544c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:25:09.625962 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efad5732-290b-40db-94c8-e068ce1d544c None admin] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:25:09.626252 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efad5732-290b-40db-94c8-e068ce1d544c None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:25:09.626532 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efad5732-290b-40db-94c8-e068ce1d544c None admin] PATH_INFO: `/v3/projects` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:25:09.627271 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efad5732-290b-40db-94c8-e068ce1d544c None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:25:09.628706 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efad5732-290b-40db-94c8-e068ce1d544c None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:25:09.643607 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:25:09.664934 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 517/1020] 10.222.0.33 () {60 vars in 1141 bytes} [Mon Jul 25 05:25:09 2022] POST /identity/v3/projects => generated 311 bytes in 215 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 05:25:10.144490 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673f3e54-7aac-42e8-99f5-5e49a427f779 None None] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:25:10.145043 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673f3e54-7aac-42e8-99f5-5e49a427f779 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:25:10.145125 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673f3e54-7aac-42e8-99f5-5e49a427f779 None None] PATH_INFO: `/` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:25:10.146621 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 504/1021] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jul 25 05:25:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:25:10.152417 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb925354-fc61-4b35-96cc-5d8031a9b90d None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:25:10.152859 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb925354-fc61-4b35-96cc-5d8031a9b90d None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:25:10.153032 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb925354-fc61-4b35-96cc-5d8031a9b90d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:25:10.170235 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-fb925354-fc61-4b35-96cc-5d8031a9b90d None None] MFA Rules not processed for user `1eae540242d94c10b0ad33960a855f08`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:25:10.171452 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-fb925354-fc61-4b35-96cc-5d8031a9b90d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:25:10.243606 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 518/1022] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jul 25 05:25:10 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 25 05:25:10.267028 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-673f3e54-7aac-42e8-99f5-5e49a427f779 None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:25:10.287398 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:25:10.372793 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-673f3e54-7aac-42e8-99f5-5e49a427f779 None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:25:10.406495 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9eba58c-c8a7-426a-be27-1c5719a295ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:25:10.407210 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9eba58c-c8a7-426a-be27-1c5719a295ab None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:25:10.407497 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9eba58c-c8a7-426a-be27-1c5719a295ab None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:25:10.407776 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9eba58c-c8a7-426a-be27-1c5719a295ab None admin] PATH_INFO: `/v3/services/image` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:25:10.408432 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9eba58c-c8a7-426a-be27-1c5719a295ab None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:25:10.409831 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9eba58c-c8a7-426a-be27-1c5719a295ab None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:25:10.421518 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application [None req-b9eba58c-c8a7-426a-be27-1c5719a295ab None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jul 25 05:25:10.421518 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:25:10.421518 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 25 05:25:10.421518 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 25 05:25:10.421518 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 25 05:25:10.421518 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 25 05:25:10.421518 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 25 05:25:10.421518 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 25 05:25:10.421518 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 25 05:25:10.421518 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Jul 25 05:25:10.421518 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 25 05:25:10.421518 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Jul 25 05:25:10.421518 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:25:10.421518 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:25:10.421518 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:25:10.421518 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:25:10.421518 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:25:10.422484 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:25:10.422484 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:25:10.422484 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:25:10.422484 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:25:10.422484 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:25:10.422484 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:25:10.422484 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 25 05:25:10.422484 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 25 05:25:10.422484 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 25 05:25:10.422484 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 25 05:25:10.422484 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:25:10.422484 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:25:10.422484 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:25:10.422484 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:25:10.422484 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:25:10.422484 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:25:10.423339 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:25:10.423339 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application with Lock( Jul 25 05:25:10.423339 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:25:10.423339 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:25:10.423339 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:25:10.423339 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:25:10.423339 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:25:10.423339 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:25:10.423339 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:25:10.423339 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:25:10.423339 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 25 05:25:10.423339 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 25 05:25:10.423339 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 25 05:25:10.423339 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Jul 25 05:25:10.425097 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 505/1023] 10.222.0.33 () {56 vars in 1105 bytes} [Mon Jul 25 05:25:10 2022] GET /identity/v3/services/image => generated 86 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:25:10.428937 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb925354-fc61-4b35-96cc-5d8031a9b90d None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:25:10.527910 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb925354-fc61-4b35-96cc-5d8031a9b90d None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:25:10.554665 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:25:10.565293 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca557f17-7d1a-49fa-83fb-4085a6f5ad58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:25:10.565968 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca557f17-7d1a-49fa-83fb-4085a6f5ad58 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:25:10.566259 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca557f17-7d1a-49fa-83fb-4085a6f5ad58 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:25:10.566543 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca557f17-7d1a-49fa-83fb-4085a6f5ad58 None admin] PATH_INFO: `/v3/services` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:25:10.567258 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca557f17-7d1a-49fa-83fb-4085a6f5ad58 None admin] RBAC: Adding query filter params (name=image) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:25:10.567571 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca557f17-7d1a-49fa-83fb-4085a6f5ad58 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:25:10.569101 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca557f17-7d1a-49fa-83fb-4085a6f5ad58 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:25:10.577095 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 519/1024] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Jul 25 05:25:10 2022] GET /identity/v3/services?name=image => generated 122 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:25:10.580592 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9eba58c-c8a7-426a-be27-1c5719a295ab None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:25:10.664145 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:25:10.684353 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9eba58c-c8a7-426a-be27-1c5719a295ab None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:25:10.717699 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a7168c8-d138-405a-a91b-2dd15201329a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:25:10.718372 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7168c8-d138-405a-a91b-2dd15201329a None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:25:10.718661 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7168c8-d138-405a-a91b-2dd15201329a None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:25:10.718948 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7168c8-d138-405a-a91b-2dd15201329a None admin] PATH_INFO: `/v3/services` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:25:10.719666 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a7168c8-d138-405a-a91b-2dd15201329a None admin] RBAC: Adding query filter params (type=image) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:25:10.719980 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a7168c8-d138-405a-a91b-2dd15201329a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:25:10.721383 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a7168c8-d138-405a-a91b-2dd15201329a None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:25:10.731798 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 506/1025] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Jul 25 05:25:10 2022] GET /identity/v3/services?type=image => generated 350 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 05:25:10.733258 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca557f17-7d1a-49fa-83fb-4085a6f5ad58 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:25:10.846282 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca557f17-7d1a-49fa-83fb-4085a6f5ad58 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:25:10.881430 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc6caaf-403a-4167-bfe0-5fc0961019c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:25:10.882460 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc6caaf-403a-4167-bfe0-5fc0961019c8 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:25:10.882992 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc6caaf-403a-4167-bfe0-5fc0961019c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:25:10.883490 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc6caaf-403a-4167-bfe0-5fc0961019c8 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:25:10.885724 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfc6caaf-403a-4167-bfe0-5fc0961019c8 None admin] RBAC: Adding query filter params (interface=public, service_id=8dbec3c12e8340d088e7bffa98d5179a, region_id=RegionOne) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:25:10.886067 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfc6caaf-403a-4167-bfe0-5fc0961019c8 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:25:10.887428 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfc6caaf-403a-4167-bfe0-5fc0961019c8 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:25:10.893195 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 520/1026] 10.222.0.33 () {56 vars in 1251 bytes} [Mon Jul 25 05:25:10 2022] GET /identity/v3/endpoints?service_id=8dbec3c12e8340d088e7bffa98d5179a&interface=public®ion_id=RegionOne => generated 194 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:25:10.898866 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a7168c8-d138-405a-a91b-2dd15201329a None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:25:11.001229 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a7168c8-d138-405a-a91b-2dd15201329a None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:25:11.034180 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6148b281-1937-4cc2-9eea-58478cd03257 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:25:11.034841 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6148b281-1937-4cc2-9eea-58478cd03257 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:25:11.035122 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6148b281-1937-4cc2-9eea-58478cd03257 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:25:11.035396 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6148b281-1937-4cc2-9eea-58478cd03257 None admin] PATH_INFO: `/v3/services` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:25:11.036071 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6148b281-1937-4cc2-9eea-58478cd03257 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:25:11.037454 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6148b281-1937-4cc2-9eea-58478cd03257 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:25:11.044363 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 507/1027] 10.222.0.33 () {56 vars in 1087 bytes} [Mon Jul 25 05:25:10 2022] GET /identity/v3/services => generated 1957 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 05:25:11.117327 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74252225-c04c-4691-8c39-116990d487c6 None None] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:25:11.117655 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74252225-c04c-4691-8c39-116990d487c6 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:25:11.117895 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74252225-c04c-4691-8c39-116990d487c6 None None] PATH_INFO: `/` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:25:11.119487 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 521/1028] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jul 25 05:25:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:25:11.125705 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8e165f-bc20-405d-9465-41e8a6e70cf3 None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:25:11.126010 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8e165f-bc20-405d-9465-41e8a6e70cf3 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:25:11.126279 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8e165f-bc20-405d-9465-41e8a6e70cf3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:25:11.144837 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-6a8e165f-bc20-405d-9465-41e8a6e70cf3 None None] MFA Rules not processed for user `1eae540242d94c10b0ad33960a855f08`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:25:11.146071 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-6a8e165f-bc20-405d-9465-41e8a6e70cf3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:25:11.149369 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:25:11.220881 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 508/1029] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jul 25 05:25:11 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 25 05:25:11.244452 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74252225-c04c-4691-8c39-116990d487c6 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:25:11.303800 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:25:11.350775 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74252225-c04c-4691-8c39-116990d487c6 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:25:11.385854 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcdd8b31-ae25-437d-a2fc-7db17414f13a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:25:11.386804 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcdd8b31-ae25-437d-a2fc-7db17414f13a None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:25:11.387084 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcdd8b31-ae25-437d-a2fc-7db17414f13a None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:25:11.387355 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcdd8b31-ae25-437d-a2fc-7db17414f13a None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:25:11.397615 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcdd8b31-ae25-437d-a2fc-7db17414f13a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 05:25:11.406109 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application [None req-dcdd8b31-ae25-437d-a2fc-7db17414f13a None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 25 05:25:11.406109 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:25:11.406109 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:25:11.406109 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:25:11.406109 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:25:11.406109 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:25:11.406109 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:25:11.406109 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:25:11.406109 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:25:11.406109 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:25:11.406109 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:25:11.406109 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:25:11.406109 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 05:25:11.406109 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 05:25:11.406109 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 05:25:11.406109 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 05:25:11.407077 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 05:25:11.407077 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 05:25:11.407077 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 05:25:11.407077 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 05:25:11.407077 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:25:11.407077 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:25:11.407077 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:25:11.407077 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:25:11.407077 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:25:11.407077 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:25:11.407077 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:25:11.407077 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application with Lock( Jul 25 05:25:11.407077 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:25:11.407077 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:25:11.407077 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:25:11.407077 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:25:11.408355 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:25:11.408355 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:25:11.408355 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:25:11.408355 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:25:11.408355 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 05:25:11.408355 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 05:25:11.408355 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 05:25:11.408355 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 05:25:11.408355 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 05:25:11.408355 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 05:25:11.408355 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 25 05:25:11.408355 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Jul 25 05:25:11.409627 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 522/1030] 10.222.0.33 () {56 vars in 1120 bytes} [Mon Jul 25 05:25:11 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 25 05:25:11.413432 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a8e165f-bc20-405d-9465-41e8a6e70cf3 None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:25:11.513200 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a8e165f-bc20-405d-9465-41e8a6e70cf3 None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:25:11.548527 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d8c2a8-449a-496f-a72d-62efa4cf1216 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:25:11.549298 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d8c2a8-449a-496f-a72d-62efa4cf1216 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:25:11.549589 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d8c2a8-449a-496f-a72d-62efa4cf1216 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:25:11.549866 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d8c2a8-449a-496f-a72d-62efa4cf1216 None admin] PATH_INFO: `/v3/roles` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:25:11.550623 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9d8c2a8-449a-496f-a72d-62efa4cf1216 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:25:11.550976 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9d8c2a8-449a-496f-a72d-62efa4cf1216 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:25:11.552293 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9d8c2a8-449a-496f-a72d-62efa4cf1216 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:25:11.566903 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 509/1031] 10.222.0.33 () {56 vars in 1115 bytes} [Mon Jul 25 05:25:11 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:25:11.568308 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcdd8b31-ae25-437d-a2fc-7db17414f13a None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:25:11.670190 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcdd8b31-ae25-437d-a2fc-7db17414f13a None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:25:11.683907 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:25:11.705491 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4996b69e-0fc8-48fd-b6cb-0869ae0cb06f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:25:11.706293 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4996b69e-0fc8-48fd-b6cb-0869ae0cb06f None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:25:11.706581 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4996b69e-0fc8-48fd-b6cb-0869ae0cb06f None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:25:11.706863 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4996b69e-0fc8-48fd-b6cb-0869ae0cb06f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:25:11.717572 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4996b69e-0fc8-48fd-b6cb-0869ae0cb06f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:25:11.719033 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4996b69e-0fc8-48fd-b6cb-0869ae0cb06f None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:25:11.736140 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application [None req-4996b69e-0fc8-48fd-b6cb-0869ae0cb06f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 05:25:11.736140 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:25:11.736140 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:25:11.736140 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:25:11.736140 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:25:11.736140 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:25:11.736140 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:25:11.736140 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:25:11.736140 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:25:11.736140 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:25:11.736140 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:25:11.736140 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:25:11.736140 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 05:25:11.736140 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 05:25:11.736140 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 05:25:11.737425 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 05:25:11.737425 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:25:11.737425 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:25:11.737425 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:25:11.737425 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:25:11.737425 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:25:11.737425 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:25:11.737425 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:25:11.737425 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application with Lock( Jul 25 05:25:11.737425 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:25:11.737425 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:25:11.737425 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:25:11.737425 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:25:11.737425 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:25:11.737425 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:25:11.737425 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:25:11.737425 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:25:11.738343 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 05:25:11.738343 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 05:25:11.738343 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 05:25:11.738343 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Jul 25 05:25:11.738343 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 523/1032] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Jul 25 05:25:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 25 05:25:11.744310 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d8c2a8-449a-496f-a72d-62efa4cf1216 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:25:11.747983 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:25:11.753040 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28bf473e-3312-46ed-858d-b9cbdd4ec38c None None] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:25:11.753334 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28bf473e-3312-46ed-858d-b9cbdd4ec38c None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:25:11.753613 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28bf473e-3312-46ed-858d-b9cbdd4ec38c None None] PATH_INFO: `/` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:25:11.755116 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 524/1033] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jul 25 05:25:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:25:11.759864 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168efe0a-528f-4a3e-8382-d7cf7bbb9cd8 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:25:11.760154 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168efe0a-528f-4a3e-8382-d7cf7bbb9cd8 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:25:11.761152 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168efe0a-528f-4a3e-8382-d7cf7bbb9cd8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:25:11.778093 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-168efe0a-528f-4a3e-8382-d7cf7bbb9cd8 None None] MFA Rules not processed for user `1eae540242d94c10b0ad33960a855f08`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:25:11.779302 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-168efe0a-528f-4a3e-8382-d7cf7bbb9cd8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:25:11.852975 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d8c2a8-449a-496f-a72d-62efa4cf1216 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:25:11.861107 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 525/1034] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jul 25 05:25:11 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 25 05:25:11.879396 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-168efe0a-528f-4a3e-8382-d7cf7bbb9cd8 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:25:11.887641 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa2cac99-a418-41b1-8186-5086b53c22a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:25:11.888307 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2cac99-a418-41b1-8186-5086b53c22a9 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:25:11.888595 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2cac99-a418-41b1-8186-5086b53c22a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:25:11.888912 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2cac99-a418-41b1-8186-5086b53c22a9 None admin] PATH_INFO: `/v3/domains` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:25:11.889647 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa2cac99-a418-41b1-8186-5086b53c22a9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:25:11.889959 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa2cac99-a418-41b1-8186-5086b53c22a9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:25:11.891325 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa2cac99-a418-41b1-8186-5086b53c22a9 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:25:11.904247 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 510/1035] 10.222.0.33 () {56 vars in 1109 bytes} [Mon Jul 25 05:25:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:25:11.909625 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa2cac99-a418-41b1-8186-5086b53c22a9 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:25:11.976443 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:25:11.984810 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:25:11.991279 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-168efe0a-528f-4a3e-8382-d7cf7bbb9cd8 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:25:12.018763 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa2cac99-a418-41b1-8186-5086b53c22a9 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:25:12.038533 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4267880-c146-48af-93e0-17573423423f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:25:12.039667 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4267880-c146-48af-93e0-17573423423f None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:25:12.040158 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4267880-c146-48af-93e0-17573423423f None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:25:12.040636 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4267880-c146-48af-93e0-17573423423f None admin] PATH_INFO: `/v3/domains/default` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:25:12.042621 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4267880-c146-48af-93e0-17573423423f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:25:12.044261 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4267880-c146-48af-93e0-17573423423f None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:25:12.046138 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 526/1036] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Jul 25 05:25:11 2022] GET /identity/v3/domains/default => generated 199 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 05:25:12.051673 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4267880-c146-48af-93e0-17573423423f None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:25:12.055960 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e055021-5ce4-4e00-9117-122132d95a82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:25:12.056751 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e055021-5ce4-4e00-9117-122132d95a82 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:25:12.057081 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e055021-5ce4-4e00-9117-122132d95a82 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:25:12.057388 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e055021-5ce4-4e00-9117-122132d95a82 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:25:12.065486 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e055021-5ce4-4e00-9117-122132d95a82 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:25:12.066992 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e055021-5ce4-4e00-9117-122132d95a82 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:25:12.081850 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application [None req-1e055021-5ce4-4e00-9117-122132d95a82 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 25 05:25:12.081850 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:25:12.081850 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:25:12.081850 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:25:12.081850 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:25:12.081850 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:25:12.081850 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:25:12.081850 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:25:12.081850 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:25:12.081850 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:25:12.081850 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:25:12.081850 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:25:12.081850 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 25 05:25:12.081850 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 25 05:25:12.081850 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 25 05:25:12.081850 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 25 05:25:12.082827 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:25:12.082827 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:25:12.082827 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 25 05:25:12.082827 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 05:25:12.082827 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 25 05:25:12.082827 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 05:25:12.082827 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:25:12.082827 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:25:12.082827 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:25:12.082827 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:25:12.082827 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:25:12.082827 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application with Lock( Jul 25 05:25:12.082827 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:25:12.082827 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:25:12.082827 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:25:12.082827 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:25:12.082827 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:25:12.082827 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:25:12.083780 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:25:12.083780 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:25:12.083780 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 25 05:25:12.083780 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 25 05:25:12.083780 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 25 05:25:12.083780 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 25 05:25:12.083780 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 25 05:25:12.083780 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 25 05:25:12.083780 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 25 05:25:12.083780 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Jul 25 05:25:12.083780 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 511/1037] 10.222.0.33 () {56 vars in 1093 bytes} [Mon Jul 25 05:25:11 2022] GET /identity/v3/users/nova => generated 82 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 25 05:25:12.089433 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e055021-5ce4-4e00-9117-122132d95a82 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:25:12.161382 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4267880-c146-48af-93e0-17573423423f None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:25:12.198275 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e055021-5ce4-4e00-9117-122132d95a82 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:25:12.198806 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c8a4ca-16fa-4834-909f-2c8b4db850a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:25:12.199386 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c8a4ca-16fa-4834-909f-2c8b4db850a0 None admin] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:25:12.199670 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c8a4ca-16fa-4834-909f-2c8b4db850a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:25:12.199944 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c8a4ca-16fa-4834-909f-2c8b4db850a0 None admin] PATH_INFO: `/v3/users` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:25:12.200658 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00c8a4ca-16fa-4834-909f-2c8b4db850a0 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:25:12.202105 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00c8a4ca-16fa-4834-909f-2c8b4db850a0 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:25:12.223708 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:25:12.226020 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 527/1038] 10.222.0.33 () {60 vars in 1132 bytes} [Mon Jul 25 05:25:12 2022] POST /identity/v3/users => generated 285 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 25 05:25:12.238987 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c81ddea9-be32-4f82-9f7d-a417713c5295 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:25:12.239684 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81ddea9-be32-4f82-9f7d-a417713c5295 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:25:12.239967 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81ddea9-be32-4f82-9f7d-a417713c5295 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:25:12.240240 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81ddea9-be32-4f82-9f7d-a417713c5295 None admin] PATH_INFO: `/v3/users` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:25:12.241159 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c81ddea9-be32-4f82-9f7d-a417713c5295 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:25:12.241463 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c81ddea9-be32-4f82-9f7d-a417713c5295 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:25:12.242941 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c81ddea9-be32-4f82-9f7d-a417713c5295 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:25:12.268217 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 512/1039] 10.222.0.33 () {56 vars in 1133 bytes} [Mon Jul 25 05:25:12 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 05:25:12.270224 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c8a4ca-16fa-4834-909f-2c8b4db850a0 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:25:12.296398 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:25:12.386775 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c8a4ca-16fa-4834-909f-2c8b4db850a0 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:25:12.423660 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b57b9db9-38fc-45c2-8503-22df2d5b6951 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:25:12.424242 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57b9db9-38fc-45c2-8503-22df2d5b6951 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:25:12.424533 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57b9db9-38fc-45c2-8503-22df2d5b6951 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:25:12.424896 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57b9db9-38fc-45c2-8503-22df2d5b6951 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:25:12.492833 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b57b9db9-38fc-45c2-8503-22df2d5b6951 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:25:12.494443 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b57b9db9-38fc-45c2-8503-22df2d5b6951 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:25:12.516495 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:25:12.602959 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 528/1040] 10.222.0.33 () {58 vars in 1282 bytes} [Mon Jul 25 05:25:12 2022] GET /identity/v3/auth/tokens => generated 1959 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 25 05:25:12.605313 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c81ddea9-be32-4f82-9f7d-a417713c5295 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:25:12.716345 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c81ddea9-be32-4f82-9f7d-a417713c5295 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:25:12.752888 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a662033c-4af5-4206-bbd1-d058511b97e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:25:12.753942 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a662033c-4af5-4206-bbd1-d058511b97e3 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:25:12.754237 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a662033c-4af5-4206-bbd1-d058511b97e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:25:12.754515 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a662033c-4af5-4206-bbd1-d058511b97e3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:25:12.765273 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a662033c-4af5-4206-bbd1-d058511b97e3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:25:12.766723 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a662033c-4af5-4206-bbd1-d058511b97e3 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:25:12.785301 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application [None req-a662033c-4af5-4206-bbd1-d058511b97e3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 05:25:12.785301 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:25:12.785301 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:25:12.785301 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:25:12.785301 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:25:12.785301 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:25:12.785301 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:25:12.785301 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:25:12.785301 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:25:12.785301 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:25:12.785301 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:25:12.785301 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:25:12.785301 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 05:25:12.785301 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 05:25:12.785301 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 05:25:12.786273 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 05:25:12.786273 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:25:12.786273 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:25:12.786273 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:25:12.786273 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:25:12.786273 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:25:12.786273 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:25:12.786273 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:25:12.786273 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application with Lock( Jul 25 05:25:12.786273 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:25:12.786273 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:25:12.786273 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:25:12.786273 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:25:12.786273 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:25:12.786273 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:25:12.786273 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:25:12.786273 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:25:12.787190 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 05:25:12.787190 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 05:25:12.787190 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 05:25:12.787190 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Jul 25 05:25:12.789096 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 513/1041] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Jul 25 05:25:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:25:12.792423 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b57b9db9-38fc-45c2-8503-22df2d5b6951 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:25:12.896112 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b57b9db9-38fc-45c2-8503-22df2d5b6951 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:25:12.917201 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:25:12.933212 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4987db02-d760-405b-9fd1-cabbd4f5a436 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:25:12.933889 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4987db02-d760-405b-9fd1-cabbd4f5a436 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:25:12.934173 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4987db02-d760-405b-9fd1-cabbd4f5a436 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:25:12.934447 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4987db02-d760-405b-9fd1-cabbd4f5a436 None admin] PATH_INFO: `/v3/domains` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:25:12.935171 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4987db02-d760-405b-9fd1-cabbd4f5a436 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:25:12.935473 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4987db02-d760-405b-9fd1-cabbd4f5a436 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:25:12.936848 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4987db02-d760-405b-9fd1-cabbd4f5a436 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:25:12.953490 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 529/1042] 10.222.0.33 () {56 vars in 1109 bytes} [Mon Jul 25 05:25:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 05:25:12.955109 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a662033c-4af5-4206-bbd1-d058511b97e3 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:25:12.986701 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:25:13.063848 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a662033c-4af5-4206-bbd1-d058511b97e3 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:25:13.097195 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e148042-4244-406b-a9f4-17ae3d4e1641 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:25:13.097971 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883b1da6-dccf-42e0-91e9-537fbaad5cf7 None None] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:25:13.098086 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e148042-4244-406b-a9f4-17ae3d4e1641 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:25:13.098249 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e148042-4244-406b-a9f4-17ae3d4e1641 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:25:13.098379 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883b1da6-dccf-42e0-91e9-537fbaad5cf7 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:25:13.098555 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e148042-4244-406b-a9f4-17ae3d4e1641 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:25:13.098652 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883b1da6-dccf-42e0-91e9-537fbaad5cf7 None None] PATH_INFO: `/` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:25:13.100049 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 530/1043] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jul 25 05:25:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:25:13.105189 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c49620-abfc-4b40-807a-8eef204b976a None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:25:13.105480 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c49620-abfc-4b40-807a-8eef204b976a None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:25:13.105762 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c49620-abfc-4b40-807a-8eef204b976a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:25:13.106374 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e148042-4244-406b-a9f4-17ae3d4e1641 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:25:13.107805 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e148042-4244-406b-a9f4-17ae3d4e1641 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:25:13.121457 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application [None req-7e148042-4244-406b-a9f4-17ae3d4e1641 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 05:25:13.121457 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:25:13.121457 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:25:13.121457 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:25:13.121457 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:25:13.121457 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:25:13.121457 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:25:13.121457 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:25:13.121457 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:25:13.121457 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:25:13.121457 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:25:13.121457 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:25:13.121457 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 25 05:25:13.121457 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 25 05:25:13.121457 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 25 05:25:13.122498 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 25 05:25:13.122498 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:25:13.122498 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:25:13.122498 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:25:13.122498 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:25:13.122498 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:25:13.122498 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:25:13.122498 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:25:13.122498 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application with Lock( Jul 25 05:25:13.122498 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:25:13.122498 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:25:13.122498 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:25:13.122498 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:25:13.122498 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:25:13.122498 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:25:13.122498 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:25:13.122498 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:25:13.128729 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 25 05:25:13.128729 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 25 05:25:13.128729 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 25 05:25:13.128729 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 25 05:25:13.128729 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 25 05:25:13.128729 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 25 05:25:13.128729 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 05:25:13.128729 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Jul 25 05:25:13.129315 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-b2c49620-abfc-4b40-807a-8eef204b976a None None] MFA Rules not processed for user `1eae540242d94c10b0ad33960a855f08`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:25:13.129436 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 514/1044] 10.222.0.33 () {56 vars in 1111 bytes} [Mon Jul 25 05:25:12 2022] GET /identity/v3/projects/service => generated 88 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 25 05:25:13.129525 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-b2c49620-abfc-4b40-807a-8eef204b976a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:25:13.130948 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e148042-4244-406b-a9f4-17ae3d4e1641 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:25:13.197100 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 531/1045] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jul 25 05:25:13 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 25 05:25:13.220234 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c49620-abfc-4b40-807a-8eef204b976a None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:25:13.233728 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:25:13.247181 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e148042-4244-406b-a9f4-17ae3d4e1641 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:25:13.286245 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01582410-3186-4f5e-a48d-d93c9f7c2fed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:25:13.286910 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01582410-3186-4f5e-a48d-d93c9f7c2fed None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:25:13.287201 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01582410-3186-4f5e-a48d-d93c9f7c2fed None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:25:13.287544 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01582410-3186-4f5e-a48d-d93c9f7c2fed None admin] PATH_INFO: `/v3/projects` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:25:13.288340 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01582410-3186-4f5e-a48d-d93c9f7c2fed None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:25:13.288709 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01582410-3186-4f5e-a48d-d93c9f7c2fed None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:25:13.290172 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01582410-3186-4f5e-a48d-d93c9f7c2fed None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:25:13.303753 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 515/1046] 10.222.0.33 () {56 vars in 1148 bytes} [Mon Jul 25 05:25:13 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 05:25:13.310107 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01582410-3186-4f5e-a48d-d93c9f7c2fed None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:25:13.327936 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c49620-abfc-4b40-807a-8eef204b976a None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:25:13.339588 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:25:13.367041 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f4d1f3-4ba3-477d-a466-19cfbaa5e31e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:25:13.367844 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f4d1f3-4ba3-477d-a466-19cfbaa5e31e None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:25:13.368130 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f4d1f3-4ba3-477d-a466-19cfbaa5e31e None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:25:13.368410 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f4d1f3-4ba3-477d-a466-19cfbaa5e31e None admin] PATH_INFO: `/v3/services/image` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:25:13.369096 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21f4d1f3-4ba3-477d-a466-19cfbaa5e31e None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:25:13.371006 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21f4d1f3-4ba3-477d-a466-19cfbaa5e31e None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:25:13.385964 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application [None req-21f4d1f3-4ba3-477d-a466-19cfbaa5e31e None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jul 25 05:25:13.385964 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:25:13.385964 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 25 05:25:13.385964 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 25 05:25:13.385964 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 25 05:25:13.385964 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 25 05:25:13.385964 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 25 05:25:13.385964 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 25 05:25:13.385964 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 25 05:25:13.385964 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Jul 25 05:25:13.385964 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 25 05:25:13.385964 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Jul 25 05:25:13.385964 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:25:13.385964 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:25:13.385964 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:25:13.385964 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:25:13.385964 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:25:13.387028 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:25:13.387028 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:25:13.387028 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:25:13.387028 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:25:13.387028 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:25:13.387028 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:25:13.387028 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 25 05:25:13.387028 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 25 05:25:13.387028 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 25 05:25:13.387028 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 25 05:25:13.387028 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:25:13.387028 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:25:13.387028 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:25:13.387028 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:25:13.387028 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:25:13.387028 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:25:13.387908 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:25:13.387908 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application with Lock( Jul 25 05:25:13.387908 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:25:13.387908 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:25:13.387908 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:25:13.387908 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:25:13.387908 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:25:13.387908 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:25:13.387908 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:25:13.387908 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:25:13.387908 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 25 05:25:13.387908 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 25 05:25:13.387908 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 25 05:25:13.387908 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Jul 25 05:25:13.388657 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 532/1047] 10.222.0.33 () {56 vars in 1105 bytes} [Mon Jul 25 05:25:13 2022] GET /identity/v3/services/image => generated 86 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 25 05:25:13.394074 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f4d1f3-4ba3-477d-a466-19cfbaa5e31e None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:25:13.422421 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01582410-3186-4f5e-a48d-d93c9f7c2fed None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:25:13.459977 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e6f57f-f391-4edf-b9cf-42b920fa7089 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:25:13.460631 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e6f57f-f391-4edf-b9cf-42b920fa7089 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:25:13.461023 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e6f57f-f391-4edf-b9cf-42b920fa7089 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:25:13.461299 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e6f57f-f391-4edf-b9cf-42b920fa7089 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:25:13.462054 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73e6f57f-f391-4edf-b9cf-42b920fa7089 None admin] RBAC: Adding query filter params (role.id=e85cbdc7a77a4fc29d71c8a3ae0f56f4, scope.project.id=92afba266cce4d1bad4134c351e74be3, user.id=1cd482a61e8a420db45bb532e5dae6e2) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:25:13.462362 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73e6f57f-f391-4edf-b9cf-42b920fa7089 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:25:13.464886 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73e6f57f-f391-4edf-b9cf-42b920fa7089 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:25:13.480814 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 516/1048] 10.222.0.33 () {56 vars in 1374 bytes} [Mon Jul 25 05:25:13 2022] GET /identity/v3/role_assignments?user.id=1cd482a61e8a420db45bb532e5dae6e2&scope.project.id=92afba266cce4d1bad4134c351e74be3&role.id=e85cbdc7a77a4fc29d71c8a3ae0f56f4 => generated 259 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 05:25:13.517849 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f4d1f3-4ba3-477d-a466-19cfbaa5e31e None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:25:13.557873 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d97de2-bb51-4720-a671-d017e2f159d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:25:13.558586 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d97de2-bb51-4720-a671-d017e2f159d7 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:25:13.558851 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d97de2-bb51-4720-a671-d017e2f159d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:25:13.559135 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d97de2-bb51-4720-a671-d017e2f159d7 None admin] PATH_INFO: `/v3/services` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:25:13.559861 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31d97de2-bb51-4720-a671-d017e2f159d7 None admin] RBAC: Adding query filter params (name=image) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:25:13.560175 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31d97de2-bb51-4720-a671-d017e2f159d7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:25:13.561593 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31d97de2-bb51-4720-a671-d017e2f159d7 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:25:13.572039 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 533/1049] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Jul 25 05:25:13 2022] GET /identity/v3/services?name=image => generated 122 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 05:25:13.574142 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e6f57f-f391-4edf-b9cf-42b920fa7089 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:25:13.593302 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:25:13.680195 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e6f57f-f391-4edf-b9cf-42b920fa7089 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:25:13.713919 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-604ac797-381a-4c65-910a-5309833057de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:25:13.714582 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604ac797-381a-4c65-910a-5309833057de None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:25:13.714863 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604ac797-381a-4c65-910a-5309833057de None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:25:13.715137 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604ac797-381a-4c65-910a-5309833057de None admin] PATH_INFO: `/v3/services` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:25:13.715845 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-604ac797-381a-4c65-910a-5309833057de None admin] RBAC: Adding query filter params (type=image) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:25:13.716151 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-604ac797-381a-4c65-910a-5309833057de None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:25:13.717533 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-604ac797-381a-4c65-910a-5309833057de None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:25:13.722580 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 517/1050] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Jul 25 05:25:13 2022] GET /identity/v3/services?type=image => generated 350 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:25:13.728018 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d97de2-bb51-4720-a671-d017e2f159d7 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:25:13.768736 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:25:13.837320 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d97de2-bb51-4720-a671-d017e2f159d7 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:25:13.871716 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b44a5a-3515-491b-9464-0f44691f6930 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:25:13.872370 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b44a5a-3515-491b-9464-0f44691f6930 None admin] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:25:13.872654 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b44a5a-3515-491b-9464-0f44691f6930 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:25:13.872961 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b44a5a-3515-491b-9464-0f44691f6930 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:25:13.873711 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96b44a5a-3515-491b-9464-0f44691f6930 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:25:13.875065 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96b44a5a-3515-491b-9464-0f44691f6930 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:25:13.900805 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 534/1051] 10.222.0.33 () {60 vars in 1144 bytes} [Mon Jul 25 05:25:13 2022] POST /identity/v3/endpoints => generated 327 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 05:25:14.346380 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be17a83e-870f-4f76-8b76-178b0878573b None None] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:25:14.346915 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be17a83e-870f-4f76-8b76-178b0878573b None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:25:14.346915 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be17a83e-870f-4f76-8b76-178b0878573b None None] PATH_INFO: `/` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:25:14.348448 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 518/1052] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jul 25 05:25:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:25:14.353497 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1fe241-8e3e-4e32-83a6-3c3b0792d100 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:25:14.353887 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1fe241-8e3e-4e32-83a6-3c3b0792d100 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:25:14.354161 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1fe241-8e3e-4e32-83a6-3c3b0792d100 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:25:14.371125 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-ba1fe241-8e3e-4e32-83a6-3c3b0792d100 None None] MFA Rules not processed for user `1eae540242d94c10b0ad33960a855f08`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:25:14.372319 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-ba1fe241-8e3e-4e32-83a6-3c3b0792d100 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:25:14.446823 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 535/1053] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jul 25 05:25:14 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 25 05:25:14.470228 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be17a83e-870f-4f76-8b76-178b0878573b None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:25:14.554344 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:25:14.578220 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be17a83e-870f-4f76-8b76-178b0878573b None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:25:14.615153 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a524fa-4a06-4b7c-bdcd-1c8dbc14269b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:25:14.615860 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a524fa-4a06-4b7c-bdcd-1c8dbc14269b None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:25:14.616146 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a524fa-4a06-4b7c-bdcd-1c8dbc14269b None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:25:14.616423 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a524fa-4a06-4b7c-bdcd-1c8dbc14269b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:25:14.627590 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1a524fa-4a06-4b7c-bdcd-1c8dbc14269b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 05:25:14.635150 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application [None req-b1a524fa-4a06-4b7c-bdcd-1c8dbc14269b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 05:25:14.635150 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:25:14.635150 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:25:14.635150 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:25:14.635150 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:25:14.635150 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:25:14.635150 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:25:14.635150 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:25:14.635150 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:25:14.635150 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:25:14.635150 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:25:14.635150 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:25:14.635150 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 05:25:14.635150 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 05:25:14.635150 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 05:25:14.635150 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 05:25:14.636185 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 05:25:14.636185 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 05:25:14.636185 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 05:25:14.636185 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 05:25:14.636185 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:25:14.636185 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:25:14.636185 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:25:14.636185 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:25:14.636185 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:25:14.636185 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:25:14.636185 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:25:14.636185 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application with Lock( Jul 25 05:25:14.636185 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:25:14.636185 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:25:14.636185 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:25:14.636185 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:25:14.637121 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:25:14.637121 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:25:14.637121 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:25:14.637121 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:25:14.637121 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 05:25:14.637121 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 05:25:14.637121 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 05:25:14.637121 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 05:25:14.637121 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 05:25:14.637121 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 05:25:14.637121 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 05:25:14.637121 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Jul 25 05:25:14.638424 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 519/1054] 10.222.0.33 () {56 vars in 1096 bytes} [Mon Jul 25 05:25:14 2022] GET /identity/v3/roles/admin => generated 83 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:25:14.642417 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba1fe241-8e3e-4e32-83a6-3c3b0792d100 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:25:14.663779 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:25:14.742557 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba1fe241-8e3e-4e32-83a6-3c3b0792d100 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:25:14.775322 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b23c49fb-bcee-45f0-a849-9efbd200e7ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:25:14.776591 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23c49fb-bcee-45f0-a849-9efbd200e7ab None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:25:14.777055 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23c49fb-bcee-45f0-a849-9efbd200e7ab None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:25:14.777141 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23c49fb-bcee-45f0-a849-9efbd200e7ab None admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:25:14.777841 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b23c49fb-bcee-45f0-a849-9efbd200e7ab None admin] RBAC: Adding query filter params (name=admin) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:25:14.778147 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b23c49fb-bcee-45f0-a849-9efbd200e7ab None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:25:14.779507 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b23c49fb-bcee-45f0-a849-9efbd200e7ab None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:25:14.787585 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 536/1055] 10.222.0.33 () {56 vars in 1099 bytes} [Mon Jul 25 05:25:14 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 05:25:14.793307 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a524fa-4a06-4b7c-bdcd-1c8dbc14269b None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:25:14.897214 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a524fa-4a06-4b7c-bdcd-1c8dbc14269b None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:25:14.929510 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5177ea8a-c8c0-434c-9051-6c691d88eb55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:25:14.930423 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5177ea8a-c8c0-434c-9051-6c691d88eb55 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:25:14.930709 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5177ea8a-c8c0-434c-9051-6c691d88eb55 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:25:14.930982 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5177ea8a-c8c0-434c-9051-6c691d88eb55 None admin] PATH_INFO: `/v3/users/05b100a5c1094566870b103f4a922409` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:25:14.942437 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:25:14.946851 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5177ea8a-c8c0-434c-9051-6c691d88eb55 None admin] RBAC: Authorizing `identity:get_user(user_id=05b100a5c1094566870b103f4a922409)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:25:14.949541 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5177ea8a-c8c0-434c-9051-6c691d88eb55 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:25:14.955749 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 520/1056] 10.222.0.33 () {56 vars in 1177 bytes} [Mon Jul 25 05:25:14 2022] GET /identity/v3/users/05b100a5c1094566870b103f4a922409 => generated 285 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:25:14.956963 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b23c49fb-bcee-45f0-a849-9efbd200e7ab None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:25:15.042438 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:25:15.063572 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b23c49fb-bcee-45f0-a849-9efbd200e7ab None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:25:15.097303 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f51d70c6-1c08-4f86-a0f5-0a4e55b58536 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:25:15.097950 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51d70c6-1c08-4f86-a0f5-0a4e55b58536 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:25:15.098234 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51d70c6-1c08-4f86-a0f5-0a4e55b58536 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:25:15.098507 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51d70c6-1c08-4f86-a0f5-0a4e55b58536 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:25:15.167269 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f51d70c6-1c08-4f86-a0f5-0a4e55b58536 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:25:15.168803 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f51d70c6-1c08-4f86-a0f5-0a4e55b58536 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:25:15.249073 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:25:15.273928 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 537/1057] 10.222.0.33 () {58 vars in 1282 bytes} [Mon Jul 25 05:25:14 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 25 05:25:15.275300 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5177ea8a-c8c0-434c-9051-6c691d88eb55 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:25:15.373292 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5177ea8a-c8c0-434c-9051-6c691d88eb55 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:25:15.407667 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-803c4f52-e3bf-4f6c-bf27-02356853fd68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:25:15.408520 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803c4f52-e3bf-4f6c-bf27-02356853fd68 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:25:15.408823 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803c4f52-e3bf-4f6c-bf27-02356853fd68 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:25:15.409102 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803c4f52-e3bf-4f6c-bf27-02356853fd68 None admin] PATH_INFO: `/v3/projects/35a0f29fd4fe4e4fba533d39b8c6b932` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:25:15.410269 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-803c4f52-e3bf-4f6c-bf27-02356853fd68 None admin] RBAC: Authorizing `identity:get_project(project_id=35a0f29fd4fe4e4fba533d39b8c6b932)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:25:15.411698 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-803c4f52-e3bf-4f6c-bf27-02356853fd68 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:25:15.418460 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 521/1058] 10.222.0.33 () {56 vars in 1186 bytes} [Mon Jul 25 05:25:15 2022] GET /identity/v3/projects/35a0f29fd4fe4e4fba533d39b8c6b932 => generated 311 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 05:25:15.419496 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f51d70c6-1c08-4f86-a0f5-0a4e55b58536 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:25:15.518481 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f51d70c6-1c08-4f86-a0f5-0a4e55b58536 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:25:15.530122 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f4c654-67d3-497c-bbc5-4730efa3a1ab None None] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:25:15.530416 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f4c654-67d3-497c-bbc5-4730efa3a1ab None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:25:15.530706 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f4c654-67d3-497c-bbc5-4730efa3a1ab None None] PATH_INFO: `/` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:25:15.532089 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 522/1059] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jul 25 05:25:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 25 05:25:15.537211 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00c6bcf-c0d4-4f2e-8ad9-e18d2d0be099 None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:25:15.537502 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00c6bcf-c0d4-4f2e-8ad9-e18d2d0be099 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:25:15.537782 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00c6bcf-c0d4-4f2e-8ad9-e18d2d0be099 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:25:15.555991 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-a00c6bcf-c0d4-4f2e-8ad9-e18d2d0be099 None None] MFA Rules not processed for user `1eae540242d94c10b0ad33960a855f08`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:25:15.556848 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f9c587-95b9-49a8-8f48-57c16beccf05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:25:15.557760 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-a00c6bcf-c0d4-4f2e-8ad9-e18d2d0be099 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:25:15.558479 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f9c587-95b9-49a8-8f48-57c16beccf05 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:25:15.558908 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f9c587-95b9-49a8-8f48-57c16beccf05 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:25:15.559403 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f9c587-95b9-49a8-8f48-57c16beccf05 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:25:15.560386 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4f9c587-95b9-49a8-8f48-57c16beccf05 None admin] RBAC: Adding query filter params (role.id=39e5c666285a42fe8fc2c3afca6260d1, scope.project.id=35a0f29fd4fe4e4fba533d39b8c6b932, user.id=05b100a5c1094566870b103f4a922409) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:25:15.560897 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4f9c587-95b9-49a8-8f48-57c16beccf05 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:25:15.562323 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4f9c587-95b9-49a8-8f48-57c16beccf05 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:25:15.576806 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 538/1060] 10.222.0.33 () {56 vars in 1374 bytes} [Mon Jul 25 05:25:15 2022] GET /identity/v3/role_assignments?user.id=05b100a5c1094566870b103f4a922409&scope.project.id=35a0f29fd4fe4e4fba533d39b8c6b932&role.id=39e5c666285a42fe8fc2c3afca6260d1 => generated 259 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:25:15.624376 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:25:15.634887 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 523/1061] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jul 25 05:25:15 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 25 05:25:15.657923 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f9c587-95b9-49a8-8f48-57c16beccf05 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:25:15.708330 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:25:15.767489 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f9c587-95b9-49a8-8f48-57c16beccf05 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:25:15.809618 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2398fe48-4278-4e53-b45a-da9089b81636 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:25:15.810473 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2398fe48-4278-4e53-b45a-da9089b81636 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:25:15.810772 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2398fe48-4278-4e53-b45a-da9089b81636 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:25:15.811051 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2398fe48-4278-4e53-b45a-da9089b81636 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:25:15.822060 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2398fe48-4278-4e53-b45a-da9089b81636 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 05:25:15.830547 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application [None req-2398fe48-4278-4e53-b45a-da9089b81636 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 25 05:25:15.830547 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:25:15.830547 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:25:15.830547 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:25:15.830547 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:25:15.830547 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:25:15.830547 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:25:15.830547 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:25:15.830547 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:25:15.830547 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:25:15.830547 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:25:15.830547 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:25:15.830547 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 05:25:15.830547 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 05:25:15.830547 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 05:25:15.830547 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 05:25:15.831495 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 05:25:15.831495 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 05:25:15.831495 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 05:25:15.831495 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 05:25:15.831495 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:25:15.831495 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:25:15.831495 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:25:15.831495 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:25:15.831495 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:25:15.831495 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:25:15.831495 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:25:15.831495 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application with Lock( Jul 25 05:25:15.831495 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:25:15.831495 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:25:15.831495 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:25:15.831495 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:25:15.832368 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:25:15.832368 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:25:15.832368 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:25:15.832368 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:25:15.832368 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 05:25:15.832368 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 05:25:15.832368 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 05:25:15.832368 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 05:25:15.832368 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 05:25:15.832368 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 05:25:15.832368 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 25 05:25:15.832368 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Jul 25 05:25:15.836859 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 539/1062] 10.222.0.33 () {56 vars in 1120 bytes} [Mon Jul 25 05:25:15 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:25:15.838232 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a00c6bcf-c0d4-4f2e-8ad9-e18d2d0be099 None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:25:15.942020 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a00c6bcf-c0d4-4f2e-8ad9-e18d2d0be099 None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:25:15.966245 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01da69a0-5667-42ca-807a-2624d0f8e4b7 None None] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:25:15.966487 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01da69a0-5667-42ca-807a-2624d0f8e4b7 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:25:15.966728 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01da69a0-5667-42ca-807a-2624d0f8e4b7 None None] PATH_INFO: `/` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:25:15.968264 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 540/1063] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jul 25 05:25:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 25 05:25:15.972979 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352681e6-373b-4657-aea4-4c93e2342264 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:25:15.973259 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352681e6-373b-4657-aea4-4c93e2342264 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:25:15.973562 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352681e6-373b-4657-aea4-4c93e2342264 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:25:15.977661 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-013c3bf2-537b-488f-a5dc-5ce509baf08b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:25:15.978341 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013c3bf2-537b-488f-a5dc-5ce509baf08b None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:25:15.978628 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013c3bf2-537b-488f-a5dc-5ce509baf08b None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:25:15.979719 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013c3bf2-537b-488f-a5dc-5ce509baf08b None admin] PATH_INFO: `/v3/roles` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:25:15.981326 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-013c3bf2-537b-488f-a5dc-5ce509baf08b None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:25:15.981660 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-013c3bf2-537b-488f-a5dc-5ce509baf08b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:25:15.983042 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-013c3bf2-537b-488f-a5dc-5ce509baf08b None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:25:15.990719 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-352681e6-373b-4657-aea4-4c93e2342264 None None] MFA Rules not processed for user `1eae540242d94c10b0ad33960a855f08`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:25:15.997054 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 524/1064] 10.222.0.33 () {56 vars in 1115 bytes} [Mon Jul 25 05:25:15 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 05:25:15.997208 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-352681e6-373b-4657-aea4-4c93e2342264 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:25:15.998755 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-013c3bf2-537b-488f-a5dc-5ce509baf08b None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:25:16.058694 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:25:16.076531 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 541/1065] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jul 25 05:25:15 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 25 05:25:16.098926 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-352681e6-373b-4657-aea4-4c93e2342264 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:25:16.106736 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-013c3bf2-537b-488f-a5dc-5ce509baf08b None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:25:16.141322 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:25:16.141904 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3616f365-26d6-443b-a9a7-bd2a082f47d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:25:16.142705 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3616f365-26d6-443b-a9a7-bd2a082f47d4 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:25:16.143062 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3616f365-26d6-443b-a9a7-bd2a082f47d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:25:16.143313 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3616f365-26d6-443b-a9a7-bd2a082f47d4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:25:16.155790 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3616f365-26d6-443b-a9a7-bd2a082f47d4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:25:16.157281 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3616f365-26d6-443b-a9a7-bd2a082f47d4 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:25:16.179936 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application [None req-3616f365-26d6-443b-a9a7-bd2a082f47d4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 05:25:16.179936 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:25:16.179936 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:25:16.179936 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:25:16.179936 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:25:16.179936 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:25:16.179936 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:25:16.179936 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:25:16.179936 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:25:16.179936 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:25:16.179936 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:25:16.179936 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:25:16.179936 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 05:25:16.179936 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 05:25:16.179936 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 05:25:16.180944 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 05:25:16.180944 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:25:16.180944 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:25:16.180944 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:25:16.180944 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:25:16.180944 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:25:16.180944 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:25:16.180944 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:25:16.180944 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application with Lock( Jul 25 05:25:16.180944 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:25:16.180944 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:25:16.180944 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:25:16.180944 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:25:16.180944 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:25:16.180944 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:25:16.180944 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:25:16.180944 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:25:16.181896 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 05:25:16.181896 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 05:25:16.181896 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 05:25:16.181896 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Jul 25 05:25:16.183354 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 525/1066] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Jul 25 05:25:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 25 05:25:16.188373 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3616f365-26d6-443b-a9a7-bd2a082f47d4 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:25:16.212109 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-352681e6-373b-4657-aea4-4c93e2342264 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:25:16.248521 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-234482a7-688e-4961-82ab-2379e64696f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:25:16.249152 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234482a7-688e-4961-82ab-2379e64696f4 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:25:16.249451 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234482a7-688e-4961-82ab-2379e64696f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:25:16.249726 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234482a7-688e-4961-82ab-2379e64696f4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:25:16.301153 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3616f365-26d6-443b-a9a7-bd2a082f47d4 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:25:16.313907 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:25:16.321717 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-234482a7-688e-4961-82ab-2379e64696f4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:25:16.323307 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-234482a7-688e-4961-82ab-2379e64696f4 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:25:16.339607 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe01f7d-d228-4bd9-96fb-0b3087e6be64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:25:16.340271 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe01f7d-d228-4bd9-96fb-0b3087e6be64 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:25:16.340560 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe01f7d-d228-4bd9-96fb-0b3087e6be64 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:25:16.340870 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe01f7d-d228-4bd9-96fb-0b3087e6be64 None admin] PATH_INFO: `/v3/domains` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:25:16.341597 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fe01f7d-d228-4bd9-96fb-0b3087e6be64 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:25:16.341956 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fe01f7d-d228-4bd9-96fb-0b3087e6be64 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:25:16.343268 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fe01f7d-d228-4bd9-96fb-0b3087e6be64 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:25:16.356037 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 526/1067] 10.222.0.33 () {56 vars in 1109 bytes} [Mon Jul 25 05:25:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 05:25:16.361847 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe01f7d-d228-4bd9-96fb-0b3087e6be64 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:25:16.366297 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:25:16.437078 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 542/1068] 10.222.0.33 () {58 vars in 1282 bytes} [Mon Jul 25 05:25:16 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 339 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 25 05:25:16.443434 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-234482a7-688e-4961-82ab-2379e64696f4 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:25:16.471458 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe01f7d-d228-4bd9-96fb-0b3087e6be64 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:25:16.507648 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33db7b36-3d57-4f1c-a380-d2c289b4c2cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:25:16.508378 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33db7b36-3d57-4f1c-a380-d2c289b4c2cd None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:25:16.508667 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33db7b36-3d57-4f1c-a380-d2c289b4c2cd None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:25:16.508979 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33db7b36-3d57-4f1c-a380-d2c289b4c2cd None admin] PATH_INFO: `/v3/users/nova` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:25:16.516419 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33db7b36-3d57-4f1c-a380-d2c289b4c2cd None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:25:16.517983 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33db7b36-3d57-4f1c-a380-d2c289b4c2cd None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:25:16.534994 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application [None req-33db7b36-3d57-4f1c-a380-d2c289b4c2cd None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 25 05:25:16.534994 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:25:16.534994 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:25:16.534994 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:25:16.534994 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:25:16.534994 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:25:16.534994 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:25:16.534994 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:25:16.534994 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:25:16.534994 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:25:16.534994 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:25:16.534994 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:25:16.534994 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 25 05:25:16.534994 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 25 05:25:16.534994 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 25 05:25:16.534994 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 25 05:25:16.535993 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:25:16.535993 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:25:16.535993 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 25 05:25:16.535993 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 05:25:16.535993 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 25 05:25:16.535993 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 05:25:16.535993 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:25:16.535993 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:25:16.535993 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:25:16.535993 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:25:16.535993 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:25:16.535993 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application with Lock( Jul 25 05:25:16.535993 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:25:16.535993 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:25:16.535993 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:25:16.535993 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:25:16.535993 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:25:16.535993 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:25:16.537021 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:25:16.537021 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:25:16.537021 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 25 05:25:16.537021 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 25 05:25:16.537021 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 25 05:25:16.537021 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 25 05:25:16.537021 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 25 05:25:16.537021 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 25 05:25:16.537021 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 25 05:25:16.537021 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Jul 25 05:25:16.537021 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 527/1069] 10.222.0.33 () {56 vars in 1093 bytes} [Mon Jul 25 05:25:16 2022] GET /identity/v3/users/nova => generated 82 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 25 05:25:16.542543 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33db7b36-3d57-4f1c-a380-d2c289b4c2cd None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:25:16.549639 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-234482a7-688e-4961-82ab-2379e64696f4 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:25:16.574485 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:25:16.589435 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68103780-307d-486b-8232-7f7c0e816aff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:25:16.590224 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68103780-307d-486b-8232-7f7c0e816aff None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:25:16.590509 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68103780-307d-486b-8232-7f7c0e816aff None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:25:16.590812 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68103780-307d-486b-8232-7f7c0e816aff None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:25:16.593491 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:25:16.601780 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68103780-307d-486b-8232-7f7c0e816aff None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:25:16.603218 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68103780-307d-486b-8232-7f7c0e816aff None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:25:16.620597 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application [None req-68103780-307d-486b-8232-7f7c0e816aff None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 05:25:16.620597 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:25:16.620597 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:25:16.620597 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:25:16.620597 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:25:16.620597 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:25:16.620597 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:25:16.620597 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:25:16.620597 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:25:16.620597 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:25:16.620597 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:25:16.620597 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:25:16.620597 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 05:25:16.620597 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 05:25:16.620597 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 05:25:16.621602 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 05:25:16.621602 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:25:16.621602 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:25:16.621602 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:25:16.621602 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:25:16.621602 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:25:16.621602 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:25:16.621602 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:25:16.621602 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application with Lock( Jul 25 05:25:16.621602 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:25:16.621602 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:25:16.621602 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:25:16.621602 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:25:16.621602 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:25:16.621602 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:25:16.621602 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:25:16.621602 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:25:16.622865 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 05:25:16.622865 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 05:25:16.622865 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 05:25:16.622865 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Jul 25 05:25:16.622865 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 543/1070] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Jul 25 05:25:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 25 05:25:16.627534 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68103780-307d-486b-8232-7f7c0e816aff None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:25:16.661409 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33db7b36-3d57-4f1c-a380-d2c289b4c2cd None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:25:16.700324 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d40ad308-1596-42cf-860e-36724a881cf0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:25:16.701025 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40ad308-1596-42cf-860e-36724a881cf0 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:25:16.701317 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40ad308-1596-42cf-860e-36724a881cf0 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:25:16.701590 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40ad308-1596-42cf-860e-36724a881cf0 None admin] PATH_INFO: `/v3/users` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:25:16.702430 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d40ad308-1596-42cf-860e-36724a881cf0 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:25:16.702739 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d40ad308-1596-42cf-860e-36724a881cf0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:25:16.704165 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d40ad308-1596-42cf-860e-36724a881cf0 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:25:16.722761 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 528/1071] 10.222.0.33 () {56 vars in 1133 bytes} [Mon Jul 25 05:25:16 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 05:25:16.728343 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d40ad308-1596-42cf-860e-36724a881cf0 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:25:16.736656 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68103780-307d-486b-8232-7f7c0e816aff None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:25:16.772201 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad4a1d1-08c0-4150-9aac-c5fc76b1f06e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:25:16.772813 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad4a1d1-08c0-4150-9aac-c5fc76b1f06e None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:25:16.773289 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad4a1d1-08c0-4150-9aac-c5fc76b1f06e None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:25:16.773398 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad4a1d1-08c0-4150-9aac-c5fc76b1f06e None admin] PATH_INFO: `/v3/domains` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:25:16.774095 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dad4a1d1-08c0-4150-9aac-c5fc76b1f06e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:25:16.774404 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dad4a1d1-08c0-4150-9aac-c5fc76b1f06e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:25:16.775789 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dad4a1d1-08c0-4150-9aac-c5fc76b1f06e None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:25:16.788345 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 544/1072] 10.222.0.33 () {56 vars in 1109 bytes} [Mon Jul 25 05:25:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 05:25:16.830647 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:25:16.844623 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d40ad308-1596-42cf-860e-36724a881cf0 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:25:16.883348 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82cd158d-b477-49a4-8885-ab93b11bd349 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:25:16.884161 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82cd158d-b477-49a4-8885-ab93b11bd349 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:25:16.884449 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82cd158d-b477-49a4-8885-ab93b11bd349 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:25:16.884729 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82cd158d-b477-49a4-8885-ab93b11bd349 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:25:16.896439 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82cd158d-b477-49a4-8885-ab93b11bd349 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:25:16.897965 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82cd158d-b477-49a4-8885-ab93b11bd349 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:25:16.918020 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application [None req-82cd158d-b477-49a4-8885-ab93b11bd349 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 05:25:16.918020 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:25:16.918020 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:25:16.918020 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:25:16.918020 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:25:16.918020 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:25:16.918020 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:25:16.918020 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:25:16.918020 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:25:16.918020 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:25:16.918020 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:25:16.918020 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:25:16.918020 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 05:25:16.918020 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 05:25:16.918020 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 05:25:16.919136 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 05:25:16.919136 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:25:16.919136 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:25:16.919136 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:25:16.919136 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:25:16.919136 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:25:16.919136 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:25:16.919136 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:25:16.919136 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application with Lock( Jul 25 05:25:16.919136 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:25:16.919136 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:25:16.919136 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:25:16.919136 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:25:16.919136 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:25:16.919136 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:25:16.919136 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:25:16.919136 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:25:16.920029 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 05:25:16.920029 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 05:25:16.920029 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 05:25:16.920029 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Jul 25 05:25:16.921121 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 529/1073] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Jul 25 05:25:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:25:16.925361 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad4a1d1-08c0-4150-9aac-c5fc76b1f06e None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:25:16.983252 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:25:17.036554 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad4a1d1-08c0-4150-9aac-c5fc76b1f06e None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:25:17.072609 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d6e0411-6081-4988-b73b-043615c73c8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:25:17.073929 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6e0411-6081-4988-b73b-043615c73c8f None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:25:17.074216 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6e0411-6081-4988-b73b-043615c73c8f None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:25:17.074498 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6e0411-6081-4988-b73b-043615c73c8f None admin] PATH_INFO: `/v3/domains` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:25:17.075234 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d6e0411-6081-4988-b73b-043615c73c8f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:25:17.075542 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d6e0411-6081-4988-b73b-043615c73c8f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:25:17.076951 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d6e0411-6081-4988-b73b-043615c73c8f None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:25:17.088811 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 545/1074] 10.222.0.33 () {56 vars in 1109 bytes} [Mon Jul 25 05:25:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:25:17.094530 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82cd158d-b477-49a4-8885-ab93b11bd349 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:25:17.192441 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82cd158d-b477-49a4-8885-ab93b11bd349 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:25:17.226257 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce2a814a-145a-4fea-a861-d34ee395ac3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:25:17.227051 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2a814a-145a-4fea-a861-d34ee395ac3e None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:25:17.227341 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2a814a-145a-4fea-a861-d34ee395ac3e None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:25:17.227622 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2a814a-145a-4fea-a861-d34ee395ac3e None admin] PATH_INFO: `/v3/projects/service` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:25:17.234369 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce2a814a-145a-4fea-a861-d34ee395ac3e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:25:17.235810 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce2a814a-145a-4fea-a861-d34ee395ac3e None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:25:17.238366 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:25:17.252314 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application [None req-ce2a814a-145a-4fea-a861-d34ee395ac3e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 05:25:17.252314 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:25:17.252314 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:25:17.252314 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:25:17.252314 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:25:17.252314 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:25:17.252314 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:25:17.252314 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:25:17.252314 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:25:17.252314 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:25:17.252314 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:25:17.252314 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:25:17.252314 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 25 05:25:17.252314 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 25 05:25:17.252314 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 25 05:25:17.253406 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 25 05:25:17.253406 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:25:17.253406 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:25:17.253406 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:25:17.253406 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:25:17.253406 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:25:17.253406 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:25:17.253406 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:25:17.253406 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application with Lock( Jul 25 05:25:17.253406 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:25:17.253406 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:25:17.253406 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:25:17.253406 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:25:17.253406 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:25:17.253406 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:25:17.253406 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:25:17.253406 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:25:17.254359 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 25 05:25:17.254359 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 25 05:25:17.254359 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 25 05:25:17.254359 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 25 05:25:17.254359 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 25 05:25:17.254359 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 25 05:25:17.254359 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 05:25:17.254359 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Jul 25 05:25:17.254359 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 530/1075] 10.222.0.33 () {56 vars in 1111 bytes} [Mon Jul 25 05:25:17 2022] GET /identity/v3/projects/service => generated 88 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:25:17.259286 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d6e0411-6081-4988-b73b-043615c73c8f None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:25:17.358366 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d6e0411-6081-4988-b73b-043615c73c8f None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:25:17.367046 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:25:17.391888 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4292fcf4-2190-47d7-8486-96eeb2c341e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:25:17.392545 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4292fcf4-2190-47d7-8486-96eeb2c341e8 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:25:17.392860 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4292fcf4-2190-47d7-8486-96eeb2c341e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:25:17.393142 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4292fcf4-2190-47d7-8486-96eeb2c341e8 None admin] PATH_INFO: `/v3/projects` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:25:17.393929 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4292fcf4-2190-47d7-8486-96eeb2c341e8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:25:17.394238 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4292fcf4-2190-47d7-8486-96eeb2c341e8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:25:17.395683 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4292fcf4-2190-47d7-8486-96eeb2c341e8 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:25:17.408864 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 546/1076] 10.222.0.33 () {56 vars in 1148 bytes} [Mon Jul 25 05:25:17 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:25:17.414971 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce2a814a-145a-4fea-a861-d34ee395ac3e None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:25:17.511714 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce2a814a-145a-4fea-a861-d34ee395ac3e None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:25:17.545847 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a75f786-c03f-419a-aa52-8f0c94116080 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:25:17.547138 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a75f786-c03f-419a-aa52-8f0c94116080 None admin] REQUEST_METHOD: `PUT` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:25:17.547483 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a75f786-c03f-419a-aa52-8f0c94116080 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:25:17.547798 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a75f786-c03f-419a-aa52-8f0c94116080 None admin] PATH_INFO: `/v3/projects/92afba266cce4d1bad4134c351e74be3/users/1cd482a61e8a420db45bb532e5dae6e2/roles/e85cbdc7a77a4fc29d71c8a3ae0f56f4` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:25:17.550611 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a75f786-c03f-419a-aa52-8f0c94116080 None admin] RBAC: Authorizing `identity:create_grant(project_id=92afba266cce4d1bad4134c351e74be3, user_id=1cd482a61e8a420db45bb532e5dae6e2, role_id=e85cbdc7a77a4fc29d71c8a3ae0f56f4)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:25:17.552044 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a75f786-c03f-419a-aa52-8f0c94116080 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:25:17.565616 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 531/1077] 10.222.0.33 () {58 vars in 1439 bytes} [Mon Jul 25 05:25:17 2022] PUT /identity/v3/projects/92afba266cce4d1bad4134c351e74be3/users/1cd482a61e8a420db45bb532e5dae6e2/roles/e85cbdc7a77a4fc29d71c8a3ae0f56f4 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 05:25:17.621856 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816d0f86-7a35-45a7-ade6-7410e08653d0 None None] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:25:17.622383 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816d0f86-7a35-45a7-ade6-7410e08653d0 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:25:17.622383 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816d0f86-7a35-45a7-ade6-7410e08653d0 None None] PATH_INFO: `/` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:25:17.623695 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 547/1078] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jul 25 05:25:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:25:17.629537 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a600ac-6abf-4d73-844a-d58eeca7fb43 None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:25:17.629831 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a600ac-6abf-4d73-844a-d58eeca7fb43 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:25:17.630112 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a600ac-6abf-4d73-844a-d58eeca7fb43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:25:17.649589 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-57a600ac-6abf-4d73-844a-d58eeca7fb43 None None] MFA Rules not processed for user `1eae540242d94c10b0ad33960a855f08`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:25:17.651419 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-57a600ac-6abf-4d73-844a-d58eeca7fb43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:25:17.714618 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:25:17.727281 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 532/1079] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jul 25 05:25:17 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 25 05:25:17.750694 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-816d0f86-7a35-45a7-ade6-7410e08653d0 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:25:17.855348 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-816d0f86-7a35-45a7-ade6-7410e08653d0 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:25:17.889907 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e14dca-fcd8-4232-8380-640c8b09b00a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:25:17.890626 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e14dca-fcd8-4232-8380-640c8b09b00a None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:25:17.890913 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e14dca-fcd8-4232-8380-640c8b09b00a None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:25:17.891194 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e14dca-fcd8-4232-8380-640c8b09b00a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:25:17.901830 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74e14dca-fcd8-4232-8380-640c8b09b00a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 05:25:17.909535 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application [None req-74e14dca-fcd8-4232-8380-640c8b09b00a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 05:25:17.909535 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:25:17.909535 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:25:17.909535 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:25:17.909535 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:25:17.909535 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:25:17.909535 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:25:17.909535 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:25:17.909535 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:25:17.909535 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:25:17.909535 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:25:17.909535 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:25:17.909535 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 05:25:17.909535 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 05:25:17.909535 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 05:25:17.909535 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 05:25:17.910540 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 05:25:17.910540 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 05:25:17.910540 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 05:25:17.910540 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 05:25:17.910540 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:25:17.910540 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:25:17.910540 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:25:17.910540 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:25:17.910540 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:25:17.910540 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:25:17.910540 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:25:17.910540 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application with Lock( Jul 25 05:25:17.910540 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:25:17.910540 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:25:17.910540 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:25:17.910540 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:25:17.911601 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:25:17.911601 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:25:17.911601 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:25:17.911601 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:25:17.911601 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 05:25:17.911601 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 05:25:17.911601 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 05:25:17.911601 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 05:25:17.911601 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 05:25:17.911601 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 05:25:17.911601 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 05:25:17.911601 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Jul 25 05:25:17.911601 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 548/1080] 10.222.0.33 () {56 vars in 1096 bytes} [Mon Jul 25 05:25:17 2022] GET /identity/v3/roles/admin => generated 83 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:25:17.917328 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a600ac-6abf-4d73-844a-d58eeca7fb43 None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:25:18.012606 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a600ac-6abf-4d73-844a-d58eeca7fb43 None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:25:18.046102 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf95b06-f99b-4eb4-b370-70936c3eddfc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:25:18.046764 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf95b06-f99b-4eb4-b370-70936c3eddfc None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:25:18.047045 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf95b06-f99b-4eb4-b370-70936c3eddfc None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:25:18.047316 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf95b06-f99b-4eb4-b370-70936c3eddfc None admin] PATH_INFO: `/v3/roles` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:25:18.048056 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cf95b06-f99b-4eb4-b370-70936c3eddfc None admin] RBAC: Adding query filter params (name=admin) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:25:18.048357 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cf95b06-f99b-4eb4-b370-70936c3eddfc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:25:18.049810 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cf95b06-f99b-4eb4-b370-70936c3eddfc None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:25:18.058441 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 533/1081] 10.222.0.33 () {56 vars in 1099 bytes} [Mon Jul 25 05:25:17 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:25:18.064754 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e14dca-fcd8-4232-8380-640c8b09b00a None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:25:18.094481 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:25:18.174287 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e14dca-fcd8-4232-8380-640c8b09b00a None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:25:18.207111 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddfa7e27-8537-47b7-ab52-2d5592dd4ff3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:25:18.207983 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddfa7e27-8537-47b7-ab52-2d5592dd4ff3 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:25:18.208268 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddfa7e27-8537-47b7-ab52-2d5592dd4ff3 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:25:18.208547 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddfa7e27-8537-47b7-ab52-2d5592dd4ff3 None admin] PATH_INFO: `/v3/users/05b100a5c1094566870b103f4a922409` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:25:18.209760 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddfa7e27-8537-47b7-ab52-2d5592dd4ff3 None admin] RBAC: Authorizing `identity:get_user(user_id=05b100a5c1094566870b103f4a922409)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:25:18.211190 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddfa7e27-8537-47b7-ab52-2d5592dd4ff3 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:25:18.212849 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 549/1082] 10.222.0.33 () {56 vars in 1177 bytes} [Mon Jul 25 05:25:18 2022] GET /identity/v3/users/05b100a5c1094566870b103f4a922409 => generated 285 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:25:18.218675 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf95b06-f99b-4eb4-b370-70936c3eddfc None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:25:18.271336 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:25:18.318221 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf95b06-f99b-4eb4-b370-70936c3eddfc None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:25:18.350472 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b215323d-7467-42a6-8d93-c92a53c5928f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:25:18.351326 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b215323d-7467-42a6-8d93-c92a53c5928f None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:25:18.351609 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b215323d-7467-42a6-8d93-c92a53c5928f None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:25:18.351885 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b215323d-7467-42a6-8d93-c92a53c5928f None admin] PATH_INFO: `/v3/projects/35a0f29fd4fe4e4fba533d39b8c6b932` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:25:18.353114 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b215323d-7467-42a6-8d93-c92a53c5928f None admin] RBAC: Authorizing `identity:get_project(project_id=35a0f29fd4fe4e4fba533d39b8c6b932)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:25:18.355405 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b215323d-7467-42a6-8d93-c92a53c5928f None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:25:18.357310 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 534/1083] 10.222.0.33 () {56 vars in 1186 bytes} [Mon Jul 25 05:25:18 2022] GET /identity/v3/projects/35a0f29fd4fe4e4fba533d39b8c6b932 => generated 311 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 05:25:18.363256 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddfa7e27-8537-47b7-ab52-2d5592dd4ff3 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:25:18.456151 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:25:18.462202 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddfa7e27-8537-47b7-ab52-2d5592dd4ff3 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:25:18.495947 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e44ea62-89cd-4af2-bec2-f51e39d38402 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:25:18.496850 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e44ea62-89cd-4af2-bec2-f51e39d38402 None admin] REQUEST_METHOD: `PUT` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:25:18.497148 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e44ea62-89cd-4af2-bec2-f51e39d38402 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:25:18.497427 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e44ea62-89cd-4af2-bec2-f51e39d38402 None admin] PATH_INFO: `/v3/projects/35a0f29fd4fe4e4fba533d39b8c6b932/users/05b100a5c1094566870b103f4a922409/roles/39e5c666285a42fe8fc2c3afca6260d1` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:25:18.499383 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e44ea62-89cd-4af2-bec2-f51e39d38402 None admin] RBAC: Authorizing `identity:create_grant(project_id=35a0f29fd4fe4e4fba533d39b8c6b932, user_id=05b100a5c1094566870b103f4a922409, role_id=39e5c666285a42fe8fc2c3afca6260d1)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:25:18.500827 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e44ea62-89cd-4af2-bec2-f51e39d38402 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:25:18.514120 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 550/1084] 10.222.0.33 () {58 vars in 1439 bytes} [Mon Jul 25 05:25:18 2022] PUT /identity/v3/projects/35a0f29fd4fe4e4fba533d39b8c6b932/users/05b100a5c1094566870b103f4a922409/roles/39e5c666285a42fe8fc2c3afca6260d1 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 05:25:19.621978 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4826f83-e7d9-46b7-97cd-e996a78cd53e None None] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:25:19.622521 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4826f83-e7d9-46b7-97cd-e996a78cd53e None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:25:19.622521 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4826f83-e7d9-46b7-97cd-e996a78cd53e None None] PATH_INFO: `/` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:25:19.623926 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 535/1085] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jul 25 05:25:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:25:19.629288 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee0b2dc-f1be-4a63-a5b3-6508fdaa9df4 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:25:19.629715 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee0b2dc-f1be-4a63-a5b3-6508fdaa9df4 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:25:19.629849 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee0b2dc-f1be-4a63-a5b3-6508fdaa9df4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:25:19.646918 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-aee0b2dc-f1be-4a63-a5b3-6508fdaa9df4 None None] MFA Rules not processed for user `1eae540242d94c10b0ad33960a855f08`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:25:19.648129 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-aee0b2dc-f1be-4a63-a5b3-6508fdaa9df4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:25:19.714024 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 551/1086] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jul 25 05:25:19 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 25 05:25:19.737305 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4826f83-e7d9-46b7-97cd-e996a78cd53e None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:25:19.839051 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4826f83-e7d9-46b7-97cd-e996a78cd53e None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:25:19.857860 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:25:19.871552 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4daf107-7c82-43c1-8292-29c7bd787ddb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:25:19.872400 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4daf107-7c82-43c1-8292-29c7bd787ddb None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:25:19.872628 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4daf107-7c82-43c1-8292-29c7bd787ddb None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:25:19.872934 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4daf107-7c82-43c1-8292-29c7bd787ddb None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:25:19.883124 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4daf107-7c82-43c1-8292-29c7bd787ddb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 05:25:19.891629 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application [None req-a4daf107-7c82-43c1-8292-29c7bd787ddb None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 25 05:25:19.891629 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:25:19.891629 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:25:19.891629 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:25:19.891629 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:25:19.891629 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:25:19.891629 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:25:19.891629 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:25:19.891629 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:25:19.891629 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:25:19.891629 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:25:19.891629 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:25:19.891629 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 05:25:19.891629 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 05:25:19.891629 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 05:25:19.891629 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 05:25:19.892563 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 05:25:19.892563 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 05:25:19.892563 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 05:25:19.892563 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 05:25:19.892563 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:25:19.892563 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:25:19.892563 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:25:19.892563 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:25:19.892563 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:25:19.892563 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:25:19.892563 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:25:19.892563 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application with Lock( Jul 25 05:25:19.892563 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:25:19.892563 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:25:19.892563 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:25:19.892563 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:25:19.893475 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:25:19.893475 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:25:19.893475 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:25:19.893475 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:25:19.893475 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 05:25:19.893475 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 05:25:19.893475 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 05:25:19.893475 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 05:25:19.893475 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 05:25:19.893475 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 05:25:19.893475 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 25 05:25:19.893475 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Jul 25 05:25:19.893475 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 536/1087] 10.222.0.33 () {56 vars in 1120 bytes} [Mon Jul 25 05:25:19 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:25:19.898846 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee0b2dc-f1be-4a63-a5b3-6508fdaa9df4 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:25:19.982217 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:25:19.998321 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee0b2dc-f1be-4a63-a5b3-6508fdaa9df4 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:25:20.032486 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa019726-4e6c-42a4-a279-7113bd62c0a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:25:20.033190 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa019726-4e6c-42a4-a279-7113bd62c0a8 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:25:20.033481 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa019726-4e6c-42a4-a279-7113bd62c0a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:25:20.033762 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa019726-4e6c-42a4-a279-7113bd62c0a8 None admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:25:20.034514 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa019726-4e6c-42a4-a279-7113bd62c0a8 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:25:20.034825 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa019726-4e6c-42a4-a279-7113bd62c0a8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:25:20.036198 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa019726-4e6c-42a4-a279-7113bd62c0a8 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:25:20.044826 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 552/1088] 10.222.0.33 () {56 vars in 1115 bytes} [Mon Jul 25 05:25:19 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:25:20.050391 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4daf107-7c82-43c1-8292-29c7bd787ddb None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:25:20.148599 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4daf107-7c82-43c1-8292-29c7bd787ddb None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:25:20.182364 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bbdea7b-3036-44ac-9551-786f6604036c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:25:20.183176 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bbdea7b-3036-44ac-9551-786f6604036c None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:25:20.183458 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bbdea7b-3036-44ac-9551-786f6604036c None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:25:20.183733 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bbdea7b-3036-44ac-9551-786f6604036c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:25:20.193770 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bbdea7b-3036-44ac-9551-786f6604036c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:25:20.195188 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bbdea7b-3036-44ac-9551-786f6604036c None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:25:20.211988 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application [None req-2bbdea7b-3036-44ac-9551-786f6604036c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 05:25:20.211988 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:25:20.211988 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:25:20.211988 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:25:20.211988 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:25:20.211988 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:25:20.211988 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:25:20.211988 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:25:20.211988 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:25:20.211988 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:25:20.211988 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:25:20.211988 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:25:20.211988 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 05:25:20.211988 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 05:25:20.211988 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 05:25:20.212943 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 05:25:20.212943 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:25:20.212943 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:25:20.212943 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:25:20.212943 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:25:20.212943 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:25:20.212943 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:25:20.212943 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:25:20.212943 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application with Lock( Jul 25 05:25:20.212943 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:25:20.212943 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:25:20.212943 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:25:20.212943 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:25:20.212943 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:25:20.212943 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:25:20.212943 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:25:20.212943 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:25:20.214368 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 05:25:20.214368 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 05:25:20.214368 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 05:25:20.214368 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Jul 25 05:25:20.214368 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 537/1089] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Jul 25 05:25:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:25:20.219491 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa019726-4e6c-42a4-a279-7113bd62c0a8 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:25:20.318054 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa019726-4e6c-42a4-a279-7113bd62c0a8 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:25:20.350831 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-103a2adc-edaa-4bef-ae16-2e75dc49d508 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:25:20.351490 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103a2adc-edaa-4bef-ae16-2e75dc49d508 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:25:20.351779 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103a2adc-edaa-4bef-ae16-2e75dc49d508 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:25:20.352059 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103a2adc-edaa-4bef-ae16-2e75dc49d508 None admin] PATH_INFO: `/v3/domains` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:25:20.352801 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-103a2adc-edaa-4bef-ae16-2e75dc49d508 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:25:20.353125 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-103a2adc-edaa-4bef-ae16-2e75dc49d508 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:25:20.355064 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-103a2adc-edaa-4bef-ae16-2e75dc49d508 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:25:20.366809 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 553/1090] 10.222.0.33 () {56 vars in 1109 bytes} [Mon Jul 25 05:25:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:25:20.372275 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bbdea7b-3036-44ac-9551-786f6604036c None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:25:20.405468 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:25:20.472536 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bbdea7b-3036-44ac-9551-786f6604036c None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:25:20.505102 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd50347-3953-4a91-a6e0-74fcb5596d1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:25:20.505885 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd50347-3953-4a91-a6e0-74fcb5596d1d None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:25:20.506169 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd50347-3953-4a91-a6e0-74fcb5596d1d None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:25:20.506444 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd50347-3953-4a91-a6e0-74fcb5596d1d None admin] PATH_INFO: `/v3/users/nova` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:25:20.514575 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bd50347-3953-4a91-a6e0-74fcb5596d1d None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:25:20.516010 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bd50347-3953-4a91-a6e0-74fcb5596d1d None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:25:20.530489 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application [None req-9bd50347-3953-4a91-a6e0-74fcb5596d1d None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 25 05:25:20.530489 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:25:20.530489 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:25:20.530489 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:25:20.530489 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:25:20.530489 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:25:20.530489 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:25:20.530489 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:25:20.530489 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:25:20.530489 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:25:20.530489 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:25:20.530489 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:25:20.530489 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 25 05:25:20.530489 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 25 05:25:20.530489 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 25 05:25:20.530489 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 25 05:25:20.531678 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:25:20.531678 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:25:20.531678 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 25 05:25:20.531678 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 05:25:20.531678 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 25 05:25:20.531678 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 05:25:20.531678 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:25:20.531678 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:25:20.531678 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:25:20.531678 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:25:20.531678 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:25:20.531678 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application with Lock( Jul 25 05:25:20.531678 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:25:20.531678 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:25:20.531678 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:25:20.531678 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:25:20.531678 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:25:20.531678 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:25:20.532969 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:25:20.532969 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:25:20.532969 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 25 05:25:20.532969 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 25 05:25:20.532969 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 25 05:25:20.532969 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 25 05:25:20.532969 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 25 05:25:20.532969 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 25 05:25:20.532969 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 25 05:25:20.532969 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Jul 25 05:25:20.533650 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac11a4f1-fbd1-4fe6-ae05-6c84f490e366 None None] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:25:20.533650 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac11a4f1-fbd1-4fe6-ae05-6c84f490e366 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:25:20.533650 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac11a4f1-fbd1-4fe6-ae05-6c84f490e366 None None] PATH_INFO: `/` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:25:20.533840 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 554/1091] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jul 25 05:25:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 25 05:25:20.536378 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 538/1092] 10.222.0.33 () {56 vars in 1093 bytes} [Mon Jul 25 05:25:20 2022] GET /identity/v3/users/nova => generated 82 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:25:20.537154 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac11a4f1-fbd1-4fe6-ae05-6c84f490e366 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:25:20.538681 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333e8d77-7a6c-4917-af91-17839aadce3a None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:25:20.538972 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333e8d77-7a6c-4917-af91-17839aadce3a None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:25:20.539254 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333e8d77-7a6c-4917-af91-17839aadce3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:25:20.549324 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:25:20.555800 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-333e8d77-7a6c-4917-af91-17839aadce3a None None] MFA Rules not processed for user `1eae540242d94c10b0ad33960a855f08`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:25:20.557065 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-333e8d77-7a6c-4917-af91-17839aadce3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:25:20.622721 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 539/1093] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jul 25 05:25:20 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 25 05:25:20.640823 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac11a4f1-fbd1-4fe6-ae05-6c84f490e366 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:25:20.646474 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-333e8d77-7a6c-4917-af91-17839aadce3a None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:25:20.670340 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:25:20.676018 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee462b8-37b5-4e4f-a084-892b16b68561 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:25:20.676703 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee462b8-37b5-4e4f-a084-892b16b68561 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:25:20.677021 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee462b8-37b5-4e4f-a084-892b16b68561 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:25:20.677301 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee462b8-37b5-4e4f-a084-892b16b68561 None admin] PATH_INFO: `/v3/users` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:25:20.678145 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ee462b8-37b5-4e4f-a084-892b16b68561 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:25:20.678491 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ee462b8-37b5-4e4f-a084-892b16b68561 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:25:20.679904 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ee462b8-37b5-4e4f-a084-892b16b68561 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:25:20.697158 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 555/1094] 10.222.0.33 () {56 vars in 1133 bytes} [Mon Jul 25 05:25:20 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:25:20.703403 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee462b8-37b5-4e4f-a084-892b16b68561 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:25:20.746764 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-333e8d77-7a6c-4917-af91-17839aadce3a None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:25:20.765885 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:25:20.780772 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-754054e7-15e3-4b6f-a47f-08a30f6fd693 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:25:20.781587 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754054e7-15e3-4b6f-a47f-08a30f6fd693 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:25:20.781873 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754054e7-15e3-4b6f-a47f-08a30f6fd693 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:25:20.782151 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754054e7-15e3-4b6f-a47f-08a30f6fd693 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:25:20.792178 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-754054e7-15e3-4b6f-a47f-08a30f6fd693 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 05:25:20.799810 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application [None req-754054e7-15e3-4b6f-a47f-08a30f6fd693 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 05:25:20.799810 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:25:20.799810 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:25:20.799810 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:25:20.799810 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:25:20.799810 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:25:20.799810 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:25:20.799810 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:25:20.799810 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:25:20.799810 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:25:20.799810 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:25:20.799810 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:25:20.799810 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 05:25:20.799810 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 05:25:20.799810 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 05:25:20.799810 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 05:25:20.801056 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 05:25:20.801056 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 05:25:20.801056 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 05:25:20.801056 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 05:25:20.801056 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:25:20.801056 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:25:20.801056 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:25:20.801056 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:25:20.801056 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:25:20.801056 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:25:20.801056 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:25:20.801056 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application with Lock( Jul 25 05:25:20.801056 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:25:20.801056 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:25:20.801056 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:25:20.801056 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:25:20.802066 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:25:20.802066 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:25:20.802066 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:25:20.802066 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:25:20.802066 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 05:25:20.802066 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 05:25:20.802066 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 05:25:20.802066 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 05:25:20.802066 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 05:25:20.802066 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 05:25:20.802066 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 05:25:20.802066 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Jul 25 05:25:20.802733 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee462b8-37b5-4e4f-a084-892b16b68561 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:25:20.805105 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 540/1095] 10.222.0.33 () {56 vars in 1096 bytes} [Mon Jul 25 05:25:20 2022] GET /identity/v3/roles/admin => generated 83 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 25 05:25:20.806909 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-754054e7-15e3-4b6f-a47f-08a30f6fd693 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:25:20.838845 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c42fa538-8534-4919-80d3-3b5b00498bc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:25:20.839505 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42fa538-8534-4919-80d3-3b5b00498bc2 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:25:20.839794 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42fa538-8534-4919-80d3-3b5b00498bc2 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:25:20.840075 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42fa538-8534-4919-80d3-3b5b00498bc2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:25:20.888877 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:25:20.907602 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-754054e7-15e3-4b6f-a47f-08a30f6fd693 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:25:20.909161 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c42fa538-8534-4919-80d3-3b5b00498bc2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:25:20.910613 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c42fa538-8534-4919-80d3-3b5b00498bc2 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:25:20.941229 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6366ecc-6c79-4932-ad1f-06d4dc825336 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:25:20.941823 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6366ecc-6c79-4932-ad1f-06d4dc825336 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:25:20.942108 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6366ecc-6c79-4932-ad1f-06d4dc825336 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:25:20.942380 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6366ecc-6c79-4932-ad1f-06d4dc825336 None admin] PATH_INFO: `/v3/roles` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:25:20.943122 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6366ecc-6c79-4932-ad1f-06d4dc825336 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:25:20.943426 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6366ecc-6c79-4932-ad1f-06d4dc825336 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:25:20.944764 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6366ecc-6c79-4932-ad1f-06d4dc825336 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:25:20.953336 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 541/1096] 10.222.0.33 () {56 vars in 1099 bytes} [Mon Jul 25 05:25:20 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 05:25:20.959029 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6366ecc-6c79-4932-ad1f-06d4dc825336 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:25:20.974214 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:25:21.009820 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 556/1097] 10.222.0.33 () {58 vars in 1282 bytes} [Mon Jul 25 05:25:20 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 25 05:25:21.015763 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c42fa538-8534-4919-80d3-3b5b00498bc2 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:25:21.055101 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6366ecc-6c79-4932-ad1f-06d4dc825336 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:25:21.088799 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-742682d9-c86f-47d4-9ff8-4278453b0c1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:25:21.089672 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742682d9-c86f-47d4-9ff8-4278453b0c1a None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:25:21.089957 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742682d9-c86f-47d4-9ff8-4278453b0c1a None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:25:21.090239 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742682d9-c86f-47d4-9ff8-4278453b0c1a None admin] PATH_INFO: `/v3/users/05b100a5c1094566870b103f4a922409` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:25:21.091481 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-742682d9-c86f-47d4-9ff8-4278453b0c1a None admin] RBAC: Authorizing `identity:get_user(user_id=05b100a5c1094566870b103f4a922409)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:25:21.092936 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-742682d9-c86f-47d4-9ff8-4278453b0c1a None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:25:21.094570 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 542/1098] 10.222.0.33 () {56 vars in 1177 bytes} [Mon Jul 25 05:25:20 2022] GET /identity/v3/users/05b100a5c1094566870b103f4a922409 => generated 285 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:25:21.100166 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-742682d9-c86f-47d4-9ff8-4278453b0c1a None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:25:21.111076 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c42fa538-8534-4919-80d3-3b5b00498bc2 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:25:21.114674 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:25:21.145418 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c80a8ca-7292-4c20-8a25-e030d214fc2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:25:21.146205 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c80a8ca-7292-4c20-8a25-e030d214fc2a None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:25:21.146492 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c80a8ca-7292-4c20-8a25-e030d214fc2a None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:25:21.146764 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c80a8ca-7292-4c20-8a25-e030d214fc2a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:25:21.157409 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c80a8ca-7292-4c20-8a25-e030d214fc2a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:25:21.158849 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c80a8ca-7292-4c20-8a25-e030d214fc2a None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:25:21.176921 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application [None req-4c80a8ca-7292-4c20-8a25-e030d214fc2a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 05:25:21.176921 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:25:21.176921 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:25:21.176921 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:25:21.176921 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:25:21.176921 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:25:21.176921 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:25:21.176921 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:25:21.176921 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:25:21.176921 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:25:21.176921 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:25:21.176921 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:25:21.176921 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 05:25:21.176921 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 05:25:21.176921 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 05:25:21.177956 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 05:25:21.177956 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:25:21.177956 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:25:21.177956 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:25:21.177956 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:25:21.177956 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:25:21.177956 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:25:21.177956 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:25:21.177956 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application with Lock( Jul 25 05:25:21.177956 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:25:21.177956 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:25:21.177956 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:25:21.177956 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:25:21.177956 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:25:21.177956 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:25:21.177956 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:25:21.177956 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:25:21.178859 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 05:25:21.178859 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 05:25:21.178859 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 05:25:21.178859 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Jul 25 05:25:21.178859 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 557/1099] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Jul 25 05:25:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 25 05:25:21.183938 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c80a8ca-7292-4c20-8a25-e030d214fc2a None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:25:21.201900 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-742682d9-c86f-47d4-9ff8-4278453b0c1a None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:25:21.203018 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:25:21.235453 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adfc8d9c-a053-4a0d-ad6d-3de9b8eaddc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:25:21.236103 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adfc8d9c-a053-4a0d-ad6d-3de9b8eaddc4 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:25:21.236391 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adfc8d9c-a053-4a0d-ad6d-3de9b8eaddc4 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:25:21.236667 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adfc8d9c-a053-4a0d-ad6d-3de9b8eaddc4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:25:21.285687 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c80a8ca-7292-4c20-8a25-e030d214fc2a None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:25:21.304186 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adfc8d9c-a053-4a0d-ad6d-3de9b8eaddc4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:25:21.305653 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adfc8d9c-a053-4a0d-ad6d-3de9b8eaddc4 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:25:21.319184 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:25:21.320850 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae32461-470e-469b-b3f5-7d5ee9f62e3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:25:21.321516 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae32461-470e-469b-b3f5-7d5ee9f62e3b None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:25:21.321801 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae32461-470e-469b-b3f5-7d5ee9f62e3b None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:25:21.322081 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae32461-470e-469b-b3f5-7d5ee9f62e3b None admin] PATH_INFO: `/v3/domains` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:25:21.323397 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ae32461-470e-469b-b3f5-7d5ee9f62e3b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:25:21.323706 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ae32461-470e-469b-b3f5-7d5ee9f62e3b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:25:21.325124 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ae32461-470e-469b-b3f5-7d5ee9f62e3b None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:25:21.336690 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 558/1100] 10.222.0.33 () {56 vars in 1109 bytes} [Mon Jul 25 05:25:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 05:25:21.343012 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae32461-470e-469b-b3f5-7d5ee9f62e3b None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:25:21.406177 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 543/1101] 10.222.0.33 () {58 vars in 1282 bytes} [Mon Jul 25 05:25:21 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 25 05:25:21.412161 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adfc8d9c-a053-4a0d-ad6d-3de9b8eaddc4 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:25:21.419860 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:25:21.441732 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae32461-470e-469b-b3f5-7d5ee9f62e3b None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:25:21.475214 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5fb00f3-b86d-4cf5-a0e4-8c64675f3ab8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:25:21.475924 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5fb00f3-b86d-4cf5-a0e4-8c64675f3ab8 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:25:21.476206 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5fb00f3-b86d-4cf5-a0e4-8c64675f3ab8 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:25:21.476482 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5fb00f3-b86d-4cf5-a0e4-8c64675f3ab8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:25:21.483065 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5fb00f3-b86d-4cf5-a0e4-8c64675f3ab8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:25:21.484493 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5fb00f3-b86d-4cf5-a0e4-8c64675f3ab8 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:25:21.499016 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application [None req-a5fb00f3-b86d-4cf5-a0e4-8c64675f3ab8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 05:25:21.499016 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:25:21.499016 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:25:21.499016 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:25:21.499016 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:25:21.499016 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:25:21.499016 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:25:21.499016 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:25:21.499016 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:25:21.499016 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:25:21.499016 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:25:21.499016 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:25:21.499016 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 25 05:25:21.499016 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 25 05:25:21.499016 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 25 05:25:21.499927 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 25 05:25:21.499927 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:25:21.499927 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:25:21.499927 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:25:21.499927 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:25:21.499927 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:25:21.499927 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:25:21.499927 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:25:21.499927 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application with Lock( Jul 25 05:25:21.499927 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:25:21.499927 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:25:21.499927 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:25:21.499927 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:25:21.499927 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:25:21.499927 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:25:21.499927 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:25:21.499927 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:25:21.500884 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 25 05:25:21.500884 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 25 05:25:21.500884 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 25 05:25:21.500884 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 25 05:25:21.500884 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 25 05:25:21.500884 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 25 05:25:21.500884 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 05:25:21.500884 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Jul 25 05:25:21.500884 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 559/1102] 10.222.0.33 () {56 vars in 1111 bytes} [Mon Jul 25 05:25:21 2022] GET /identity/v3/projects/service => generated 88 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 25 05:25:21.505862 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5fb00f3-b86d-4cf5-a0e4-8c64675f3ab8 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:25:21.507385 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adfc8d9c-a053-4a0d-ad6d-3de9b8eaddc4 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:25:21.520692 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:25:21.542547 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a4ee3f-f25e-4bd5-bb97-44bd557bb206 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:25:21.543408 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a4ee3f-f25e-4bd5-bb97-44bd557bb206 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:25:21.543697 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a4ee3f-f25e-4bd5-bb97-44bd557bb206 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:25:21.543977 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a4ee3f-f25e-4bd5-bb97-44bd557bb206 None admin] PATH_INFO: `/v3/projects/35a0f29fd4fe4e4fba533d39b8c6b932` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:25:21.545205 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15a4ee3f-f25e-4bd5-bb97-44bd557bb206 None admin] RBAC: Authorizing `identity:get_project(project_id=35a0f29fd4fe4e4fba533d39b8c6b932)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:25:21.546644 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15a4ee3f-f25e-4bd5-bb97-44bd557bb206 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:25:21.548487 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 544/1103] 10.222.0.33 () {56 vars in 1186 bytes} [Mon Jul 25 05:25:21 2022] GET /identity/v3/projects/35a0f29fd4fe4e4fba533d39b8c6b932 => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:25:21.554657 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a4ee3f-f25e-4bd5-bb97-44bd557bb206 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:25:21.604891 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5fb00f3-b86d-4cf5-a0e4-8c64675f3ab8 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:25:21.639619 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c93c5253-7626-4afe-a75f-5f9d88a76190 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:25:21.640277 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93c5253-7626-4afe-a75f-5f9d88a76190 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:25:21.640560 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93c5253-7626-4afe-a75f-5f9d88a76190 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:25:21.640866 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93c5253-7626-4afe-a75f-5f9d88a76190 None admin] PATH_INFO: `/v3/projects` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:25:21.641657 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c93c5253-7626-4afe-a75f-5f9d88a76190 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:25:21.641964 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c93c5253-7626-4afe-a75f-5f9d88a76190 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:25:21.643374 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c93c5253-7626-4afe-a75f-5f9d88a76190 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:25:21.650214 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:25:21.652773 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a4ee3f-f25e-4bd5-bb97-44bd557bb206 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:25:21.655955 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 560/1104] 10.222.0.33 () {56 vars in 1148 bytes} [Mon Jul 25 05:25:21 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 05:25:21.661751 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c93c5253-7626-4afe-a75f-5f9d88a76190 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:25:21.687127 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d860b45-fafb-4fed-a266-b6cc88b9867e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:25:21.687776 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d860b45-fafb-4fed-a266-b6cc88b9867e None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:25:21.688058 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d860b45-fafb-4fed-a266-b6cc88b9867e None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:25:21.688399 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d860b45-fafb-4fed-a266-b6cc88b9867e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:25:21.689198 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d860b45-fafb-4fed-a266-b6cc88b9867e None admin] RBAC: Adding query filter params (role.id=39e5c666285a42fe8fc2c3afca6260d1, scope.project.id=35a0f29fd4fe4e4fba533d39b8c6b932, user.id=05b100a5c1094566870b103f4a922409) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:25:21.689772 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d860b45-fafb-4fed-a266-b6cc88b9867e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:25:21.691187 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d860b45-fafb-4fed-a266-b6cc88b9867e None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:25:21.696935 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 545/1105] 10.222.0.33 () {56 vars in 1374 bytes} [Mon Jul 25 05:25:21 2022] GET /identity/v3/role_assignments?user.id=05b100a5c1094566870b103f4a922409&scope.project.id=35a0f29fd4fe4e4fba533d39b8c6b932&role.id=39e5c666285a42fe8fc2c3afca6260d1 => generated 608 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 05:25:21.763372 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c93c5253-7626-4afe-a75f-5f9d88a76190 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:25:21.798723 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3626fa2b-22fc-4820-9fab-fc5fb0c7939e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:25:21.799395 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3626fa2b-22fc-4820-9fab-fc5fb0c7939e None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:25:21.799679 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3626fa2b-22fc-4820-9fab-fc5fb0c7939e None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:25:21.799956 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3626fa2b-22fc-4820-9fab-fc5fb0c7939e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:25:21.800835 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3626fa2b-22fc-4820-9fab-fc5fb0c7939e None admin] RBAC: Adding query filter params (role.id=e85cbdc7a77a4fc29d71c8a3ae0f56f4, scope.project.id=92afba266cce4d1bad4134c351e74be3, user.id=1cd482a61e8a420db45bb532e5dae6e2) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:25:21.801151 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3626fa2b-22fc-4820-9fab-fc5fb0c7939e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:25:21.802606 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3626fa2b-22fc-4820-9fab-fc5fb0c7939e None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:25:21.809492 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 561/1106] 10.222.0.33 () {56 vars in 1374 bytes} [Mon Jul 25 05:25:21 2022] GET /identity/v3/role_assignments?user.id=1cd482a61e8a420db45bb532e5dae6e2&scope.project.id=92afba266cce4d1bad4134c351e74be3&role.id=e85cbdc7a77a4fc29d71c8a3ae0f56f4 => generated 608 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 05:25:23.754130 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3322cbe-b171-44f1-9447-2820f327b70a None None] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:25:23.754769 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3322cbe-b171-44f1-9447-2820f327b70a None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:25:23.754769 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3322cbe-b171-44f1-9447-2820f327b70a None None] PATH_INFO: `/` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:25:23.756045 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 546/1107] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jul 25 05:25:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:25:23.761899 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4c3834-6918-435d-a798-63b58e45a5b7 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:25:23.762309 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4c3834-6918-435d-a798-63b58e45a5b7 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:25:23.762461 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4c3834-6918-435d-a798-63b58e45a5b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:25:23.778840 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-4f4c3834-6918-435d-a798-63b58e45a5b7 None None] MFA Rules not processed for user `1eae540242d94c10b0ad33960a855f08`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:25:23.780046 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-4f4c3834-6918-435d-a798-63b58e45a5b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:25:23.835634 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:25:23.846097 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 562/1108] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jul 25 05:25:23 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 25 05:25:23.870986 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3322cbe-b171-44f1-9447-2820f327b70a None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:25:23.916373 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:25:23.971366 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3322cbe-b171-44f1-9447-2820f327b70a None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:25:24.004559 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b11490c0-eff8-43f4-94da-5c09a5b959e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:25:24.005328 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11490c0-eff8-43f4-94da-5c09a5b959e1 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:25:24.005618 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11490c0-eff8-43f4-94da-5c09a5b959e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:25:24.005899 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11490c0-eff8-43f4-94da-5c09a5b959e1 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:25:24.007073 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b11490c0-eff8-43f4-94da-5c09a5b959e1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:25:24.008509 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b11490c0-eff8-43f4-94da-5c09a5b959e1 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:25:24.010692 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 547/1109] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Jul 25 05:25:23 2022] GET /identity/v3/domains/default => generated 199 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:25:24.016489 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f4c3834-6918-435d-a798-63b58e45a5b7 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:25:24.113640 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f4c3834-6918-435d-a798-63b58e45a5b7 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:25:24.147067 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-135b0364-7b06-4b9f-ac73-e60fd2fea645 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:25:24.147738 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135b0364-7b06-4b9f-ac73-e60fd2fea645 None admin] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:25:24.148025 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135b0364-7b06-4b9f-ac73-e60fd2fea645 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:25:24.148306 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135b0364-7b06-4b9f-ac73-e60fd2fea645 None admin] PATH_INFO: `/v3/users` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:25:24.149056 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-135b0364-7b06-4b9f-ac73-e60fd2fea645 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:25:24.150476 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-135b0364-7b06-4b9f-ac73-e60fd2fea645 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:25:24.170216 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 563/1110] 10.222.0.33 () {60 vars in 1132 bytes} [Mon Jul 25 05:25:24 2022] POST /identity/v3/users => generated 286 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 05:25:26.224878 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04eedfa6-62e1-4cf6-b4c6-8b04c82184be None None] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:25:26.225396 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04eedfa6-62e1-4cf6-b4c6-8b04c82184be None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:25:26.225396 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04eedfa6-62e1-4cf6-b4c6-8b04c82184be None None] PATH_INFO: `/` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:25:26.226734 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 548/1111] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jul 25 05:25:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:25:26.232066 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359f7ab2-be41-4ebb-b979-1abc8c086f07 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:25:26.232516 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359f7ab2-be41-4ebb-b979-1abc8c086f07 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:25:26.232645 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359f7ab2-be41-4ebb-b979-1abc8c086f07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:25:26.249915 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-359f7ab2-be41-4ebb-b979-1abc8c086f07 None None] MFA Rules not processed for user `1eae540242d94c10b0ad33960a855f08`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:25:26.251109 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-359f7ab2-be41-4ebb-b979-1abc8c086f07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:25:26.293173 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:25:26.317014 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 564/1112] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jul 25 05:25:26 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 25 05:25:26.341208 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04eedfa6-62e1-4cf6-b4c6-8b04c82184be None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:25:26.440230 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04eedfa6-62e1-4cf6-b4c6-8b04c82184be None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:25:26.453028 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:25:26.473390 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d067a23-43c9-4f9d-aa89-6b392f543874 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:25:26.474233 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d067a23-43c9-4f9d-aa89-6b392f543874 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:25:26.474520 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d067a23-43c9-4f9d-aa89-6b392f543874 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:25:26.474797 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d067a23-43c9-4f9d-aa89-6b392f543874 None admin] PATH_INFO: `/v3/roles/02aba1f7cbdf4d6595e18af7fb5727e0` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:25:26.476308 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d067a23-43c9-4f9d-aa89-6b392f543874 None admin] RBAC: Authorizing `identity:get_role(role_id=02aba1f7cbdf4d6595e18af7fb5727e0)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:25:26.477641 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d067a23-43c9-4f9d-aa89-6b392f543874 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:25:26.479770 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 549/1113] 10.222.0.33 () {56 vars in 1177 bytes} [Mon Jul 25 05:25:26 2022] GET /identity/v3/roles/02aba1f7cbdf4d6595e18af7fb5727e0 => generated 223 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:25:26.485519 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-359f7ab2-be41-4ebb-b979-1abc8c086f07 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:25:26.580442 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-359f7ab2-be41-4ebb-b979-1abc8c086f07 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:25:26.613048 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e41bcb90-aa24-4456-9161-3120399a5332 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:25:26.613909 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41bcb90-aa24-4456-9161-3120399a5332 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:25:26.614194 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41bcb90-aa24-4456-9161-3120399a5332 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:25:26.614472 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41bcb90-aa24-4456-9161-3120399a5332 None admin] PATH_INFO: `/v3/users/fc3d219dd8ff4a18b3e34f1961c1e0bc` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:25:26.627698 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e41bcb90-aa24-4456-9161-3120399a5332 None admin] RBAC: Authorizing `identity:get_user(user_id=fc3d219dd8ff4a18b3e34f1961c1e0bc)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:25:26.629162 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e41bcb90-aa24-4456-9161-3120399a5332 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:25:26.630787 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 565/1114] 10.222.0.33 () {56 vars in 1177 bytes} [Mon Jul 25 05:25:26 2022] GET /identity/v3/users/fc3d219dd8ff4a18b3e34f1961c1e0bc => generated 286 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:25:26.636427 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d067a23-43c9-4f9d-aa89-6b392f543874 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:25:26.732801 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d067a23-43c9-4f9d-aa89-6b392f543874 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:25:26.764428 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-266f0761-6e4c-4e8a-b11d-4fd5b20089b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:25:26.765088 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266f0761-6e4c-4e8a-b11d-4fd5b20089b8 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:25:26.765371 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266f0761-6e4c-4e8a-b11d-4fd5b20089b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:25:26.765646 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266f0761-6e4c-4e8a-b11d-4fd5b20089b8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:25:26.811810 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:25:26.830949 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-266f0761-6e4c-4e8a-b11d-4fd5b20089b8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:25:26.832384 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-266f0761-6e4c-4e8a-b11d-4fd5b20089b8 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:25:26.925549 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 550/1115] 10.222.0.33 () {58 vars in 1282 bytes} [Mon Jul 25 05:25:26 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 290 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 25 05:25:26.931505 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e41bcb90-aa24-4456-9161-3120399a5332 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:25:26.965724 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:25:27.027875 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e41bcb90-aa24-4456-9161-3120399a5332 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:25:27.060317 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4130e9d8-d0c8-4a34-b30c-1361d1b3aebe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:25:27.061191 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4130e9d8-d0c8-4a34-b30c-1361d1b3aebe None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:25:27.061477 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4130e9d8-d0c8-4a34-b30c-1361d1b3aebe None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:25:27.061752 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4130e9d8-d0c8-4a34-b30c-1361d1b3aebe None admin] PATH_INFO: `/v3/projects/35a0f29fd4fe4e4fba533d39b8c6b932` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:25:27.062918 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4130e9d8-d0c8-4a34-b30c-1361d1b3aebe None admin] RBAC: Authorizing `identity:get_project(project_id=35a0f29fd4fe4e4fba533d39b8c6b932)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:25:27.064345 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4130e9d8-d0c8-4a34-b30c-1361d1b3aebe None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:25:27.066315 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 566/1116] 10.222.0.33 () {56 vars in 1186 bytes} [Mon Jul 25 05:25:26 2022] GET /identity/v3/projects/35a0f29fd4fe4e4fba533d39b8c6b932 => generated 311 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:25:27.072417 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-266f0761-6e4c-4e8a-b11d-4fd5b20089b8 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:25:27.150033 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:25:27.169131 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-266f0761-6e4c-4e8a-b11d-4fd5b20089b8 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:25:27.202163 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a233b8f-5882-4804-accb-cd07050f663b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:25:27.202803 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a233b8f-5882-4804-accb-cd07050f663b None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:25:27.203089 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a233b8f-5882-4804-accb-cd07050f663b None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:25:27.203368 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a233b8f-5882-4804-accb-cd07050f663b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:25:27.204200 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a233b8f-5882-4804-accb-cd07050f663b None admin] RBAC: Adding query filter params (role.id=02aba1f7cbdf4d6595e18af7fb5727e0, scope.project.id=35a0f29fd4fe4e4fba533d39b8c6b932, user.id=fc3d219dd8ff4a18b3e34f1961c1e0bc) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:25:27.204515 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a233b8f-5882-4804-accb-cd07050f663b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:25:27.205977 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a233b8f-5882-4804-accb-cd07050f663b None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:25:27.211258 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 551/1117] 10.222.0.33 () {56 vars in 1374 bytes} [Mon Jul 25 05:25:27 2022] GET /identity/v3/role_assignments?user.id=fc3d219dd8ff4a18b3e34f1961c1e0bc&scope.project.id=35a0f29fd4fe4e4fba533d39b8c6b932&role.id=02aba1f7cbdf4d6595e18af7fb5727e0 => generated 259 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:25:29.204921 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4869af-e3c6-4a22-b346-713a07fb45c9 None None] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:25:29.205493 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4869af-e3c6-4a22-b346-713a07fb45c9 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:25:29.205493 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4869af-e3c6-4a22-b346-713a07fb45c9 None None] PATH_INFO: `/` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:25:29.206701 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 567/1118] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jul 25 05:25:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:25:29.211871 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4104fb3-4fdd-4eed-a71f-e163cb605196 None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:25:29.212304 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4104fb3-4fdd-4eed-a71f-e163cb605196 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:25:29.212404 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4104fb3-4fdd-4eed-a71f-e163cb605196 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:25:29.230042 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-e4104fb3-4fdd-4eed-a71f-e163cb605196 None None] MFA Rules not processed for user `1eae540242d94c10b0ad33960a855f08`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:25:29.231216 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-e4104fb3-4fdd-4eed-a71f-e163cb605196 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:25:29.295797 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 552/1119] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jul 25 05:25:29 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 25 05:25:29.319129 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a4869af-e3c6-4a22-b346-713a07fb45c9 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:25:29.416235 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a4869af-e3c6-4a22-b346-713a07fb45c9 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:25:29.419002 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:25:29.450898 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee35cb6e-90f3-423d-b80d-35d446ba041f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:25:29.451733 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee35cb6e-90f3-423d-b80d-35d446ba041f None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:25:29.452011 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee35cb6e-90f3-423d-b80d-35d446ba041f None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:25:29.452280 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee35cb6e-90f3-423d-b80d-35d446ba041f None admin] PATH_INFO: `/v3/roles/02aba1f7cbdf4d6595e18af7fb5727e0` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:25:29.453762 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee35cb6e-90f3-423d-b80d-35d446ba041f None admin] RBAC: Authorizing `identity:get_role(role_id=02aba1f7cbdf4d6595e18af7fb5727e0)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:25:29.455165 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee35cb6e-90f3-423d-b80d-35d446ba041f None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:25:29.456483 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 568/1120] 10.222.0.33 () {56 vars in 1177 bytes} [Mon Jul 25 05:25:29 2022] GET /identity/v3/roles/02aba1f7cbdf4d6595e18af7fb5727e0 => generated 223 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:25:29.462197 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4104fb3-4fdd-4eed-a71f-e163cb605196 None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:25:29.535904 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:25:29.561719 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4104fb3-4fdd-4eed-a71f-e163cb605196 None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:25:29.594856 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d952e3-80d6-468a-ac62-68928fbc88ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:25:29.595716 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d952e3-80d6-468a-ac62-68928fbc88ac None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:25:29.596000 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d952e3-80d6-468a-ac62-68928fbc88ac None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:25:29.596278 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d952e3-80d6-468a-ac62-68928fbc88ac None admin] PATH_INFO: `/v3/users/fc3d219dd8ff4a18b3e34f1961c1e0bc` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:25:29.597459 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85d952e3-80d6-468a-ac62-68928fbc88ac None admin] RBAC: Authorizing `identity:get_user(user_id=fc3d219dd8ff4a18b3e34f1961c1e0bc)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:25:29.598883 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85d952e3-80d6-468a-ac62-68928fbc88ac None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:25:29.600499 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 553/1121] 10.222.0.33 () {56 vars in 1177 bytes} [Mon Jul 25 05:25:29 2022] GET /identity/v3/users/fc3d219dd8ff4a18b3e34f1961c1e0bc => generated 286 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:25:29.606697 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee35cb6e-90f3-423d-b80d-35d446ba041f None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:25:29.702677 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee35cb6e-90f3-423d-b80d-35d446ba041f None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:25:29.736565 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f98a48-417f-47b0-8b63-c98d0cb97067 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:25:29.737378 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f98a48-417f-47b0-8b63-c98d0cb97067 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:25:29.737660 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f98a48-417f-47b0-8b63-c98d0cb97067 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:25:29.737931 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f98a48-417f-47b0-8b63-c98d0cb97067 None admin] PATH_INFO: `/v3/projects/35a0f29fd4fe4e4fba533d39b8c6b932` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:25:29.739092 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6f98a48-417f-47b0-8b63-c98d0cb97067 None admin] RBAC: Authorizing `identity:get_project(project_id=35a0f29fd4fe4e4fba533d39b8c6b932)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:25:29.740493 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6f98a48-417f-47b0-8b63-c98d0cb97067 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:25:29.742181 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 569/1122] 10.222.0.33 () {56 vars in 1186 bytes} [Mon Jul 25 05:25:29 2022] GET /identity/v3/projects/35a0f29fd4fe4e4fba533d39b8c6b932 => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:25:29.748489 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d952e3-80d6-468a-ac62-68928fbc88ac None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:25:29.844963 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d952e3-80d6-468a-ac62-68928fbc88ac None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:25:29.880128 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52824b5d-e735-4f93-b50c-4234b1cbfdb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:25:29.881034 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52824b5d-e735-4f93-b50c-4234b1cbfdb6 None admin] REQUEST_METHOD: `PUT` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:25:29.881319 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52824b5d-e735-4f93-b50c-4234b1cbfdb6 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:25:29.881594 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52824b5d-e735-4f93-b50c-4234b1cbfdb6 None admin] PATH_INFO: `/v3/projects/35a0f29fd4fe4e4fba533d39b8c6b932/users/fc3d219dd8ff4a18b3e34f1961c1e0bc/roles/02aba1f7cbdf4d6595e18af7fb5727e0` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:25:29.883793 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52824b5d-e735-4f93-b50c-4234b1cbfdb6 None admin] RBAC: Authorizing `identity:create_grant(project_id=35a0f29fd4fe4e4fba533d39b8c6b932, user_id=fc3d219dd8ff4a18b3e34f1961c1e0bc, role_id=02aba1f7cbdf4d6595e18af7fb5727e0)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:25:29.885357 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52824b5d-e735-4f93-b50c-4234b1cbfdb6 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:25:29.898259 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 554/1123] 10.222.0.33 () {58 vars in 1439 bytes} [Mon Jul 25 05:25:29 2022] PUT /identity/v3/projects/35a0f29fd4fe4e4fba533d39b8c6b932/users/fc3d219dd8ff4a18b3e34f1961c1e0bc/roles/02aba1f7cbdf4d6595e18af7fb5727e0 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 05:25:31.911725 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10f48da-29fe-427d-9a92-1d713c73ee63 None None] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:25:31.912240 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10f48da-29fe-427d-9a92-1d713c73ee63 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:25:31.912346 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10f48da-29fe-427d-9a92-1d713c73ee63 None None] PATH_INFO: `/` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:25:31.913690 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 570/1124] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jul 25 05:25:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:25:31.919411 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82c4ff2-91eb-4fd2-842e-5e8a4ee3c55c None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:25:31.919858 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82c4ff2-91eb-4fd2-842e-5e8a4ee3c55c None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:25:31.919964 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82c4ff2-91eb-4fd2-842e-5e8a4ee3c55c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:25:31.924916 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:25:31.938956 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-e82c4ff2-91eb-4fd2-842e-5e8a4ee3c55c None None] MFA Rules not processed for user `1eae540242d94c10b0ad33960a855f08`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:25:31.940390 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-e82c4ff2-91eb-4fd2-842e-5e8a4ee3c55c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:25:32.003379 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 555/1125] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jul 25 05:25:31 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 25 05:25:32.027472 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b10f48da-29fe-427d-9a92-1d713c73ee63 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:25:32.070003 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:25:32.127678 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b10f48da-29fe-427d-9a92-1d713c73ee63 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:25:32.160909 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2afe13d-e9d6-426a-acb3-1be0481692a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:25:32.161754 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2afe13d-e9d6-426a-acb3-1be0481692a9 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:25:32.162039 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2afe13d-e9d6-426a-acb3-1be0481692a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:25:32.162317 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2afe13d-e9d6-426a-acb3-1be0481692a9 None admin] PATH_INFO: `/v3/roles/02aba1f7cbdf4d6595e18af7fb5727e0` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:25:32.163826 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2afe13d-e9d6-426a-acb3-1be0481692a9 None admin] RBAC: Authorizing `identity:get_role(role_id=02aba1f7cbdf4d6595e18af7fb5727e0)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:25:32.165224 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2afe13d-e9d6-426a-acb3-1be0481692a9 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:25:32.167382 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 571/1126] 10.222.0.33 () {56 vars in 1177 bytes} [Mon Jul 25 05:25:32 2022] GET /identity/v3/roles/02aba1f7cbdf4d6595e18af7fb5727e0 => generated 223 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:25:32.173127 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e82c4ff2-91eb-4fd2-842e-5e8a4ee3c55c None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:25:32.270331 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e82c4ff2-91eb-4fd2-842e-5e8a4ee3c55c None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:25:32.299437 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:25:32.304726 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a9c7461-987a-4d1d-9aa4-43e014c5fafc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:25:32.305620 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9c7461-987a-4d1d-9aa4-43e014c5fafc None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:25:32.305909 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9c7461-987a-4d1d-9aa4-43e014c5fafc None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:25:32.306189 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9c7461-987a-4d1d-9aa4-43e014c5fafc None admin] PATH_INFO: `/v3/users/fc3d219dd8ff4a18b3e34f1961c1e0bc` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:25:32.307418 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a9c7461-987a-4d1d-9aa4-43e014c5fafc None admin] RBAC: Authorizing `identity:get_user(user_id=fc3d219dd8ff4a18b3e34f1961c1e0bc)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:25:32.308948 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a9c7461-987a-4d1d-9aa4-43e014c5fafc None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:25:32.310591 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 556/1127] 10.222.0.33 () {56 vars in 1177 bytes} [Mon Jul 25 05:25:32 2022] GET /identity/v3/users/fc3d219dd8ff4a18b3e34f1961c1e0bc => generated 286 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:25:32.316563 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2afe13d-e9d6-426a-acb3-1be0481692a9 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:25:32.412626 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2afe13d-e9d6-426a-acb3-1be0481692a9 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:25:32.421503 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:25:32.445688 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b206e1-1bf2-4611-9a9f-c72fbd722fbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:25:32.446321 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b206e1-1bf2-4611-9a9f-c72fbd722fbd None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:25:32.446605 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b206e1-1bf2-4611-9a9f-c72fbd722fbd None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:25:32.446909 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b206e1-1bf2-4611-9a9f-c72fbd722fbd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:25:32.510670 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8b206e1-1bf2-4611-9a9f-c72fbd722fbd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:25:32.512085 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8b206e1-1bf2-4611-9a9f-c72fbd722fbd None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:25:32.606342 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 572/1128] 10.222.0.33 () {58 vars in 1282 bytes} [Mon Jul 25 05:25:32 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 290 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 25 05:25:32.612193 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a9c7461-987a-4d1d-9aa4-43e014c5fafc None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:25:32.706989 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a9c7461-987a-4d1d-9aa4-43e014c5fafc None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:25:32.739457 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b4d532-6054-4a6b-b88a-a27d913d24f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:25:32.740302 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b4d532-6054-4a6b-b88a-a27d913d24f3 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:25:32.740582 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b4d532-6054-4a6b-b88a-a27d913d24f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:25:32.740881 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b4d532-6054-4a6b-b88a-a27d913d24f3 None admin] PATH_INFO: `/v3/projects/35a0f29fd4fe4e4fba533d39b8c6b932` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:25:32.742037 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88b4d532-6054-4a6b-b88a-a27d913d24f3 None admin] RBAC: Authorizing `identity:get_project(project_id=35a0f29fd4fe4e4fba533d39b8c6b932)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:25:32.743449 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88b4d532-6054-4a6b-b88a-a27d913d24f3 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:25:32.745819 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 557/1129] 10.222.0.33 () {56 vars in 1186 bytes} [Mon Jul 25 05:25:32 2022] GET /identity/v3/projects/35a0f29fd4fe4e4fba533d39b8c6b932 => generated 311 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:25:32.751874 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b206e1-1bf2-4611-9a9f-c72fbd722fbd None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:25:32.762253 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:25:32.849131 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b206e1-1bf2-4611-9a9f-c72fbd722fbd None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:25:32.881155 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a94f23-25b5-49b7-9921-53bda5e1b6db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:25:32.881796 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a94f23-25b5-49b7-9921-53bda5e1b6db None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:25:32.882079 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a94f23-25b5-49b7-9921-53bda5e1b6db None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:25:32.882354 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a94f23-25b5-49b7-9921-53bda5e1b6db None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:25:32.883174 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0a94f23-25b5-49b7-9921-53bda5e1b6db None admin] RBAC: Adding query filter params (role.id=02aba1f7cbdf4d6595e18af7fb5727e0, scope.project.id=35a0f29fd4fe4e4fba533d39b8c6b932, user.id=fc3d219dd8ff4a18b3e34f1961c1e0bc) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:25:32.883486 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0a94f23-25b5-49b7-9921-53bda5e1b6db None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:25:32.884968 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0a94f23-25b5-49b7-9921-53bda5e1b6db None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:25:32.890594 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 573/1130] 10.222.0.33 () {56 vars in 1374 bytes} [Mon Jul 25 05:25:32 2022] GET /identity/v3/role_assignments?user.id=fc3d219dd8ff4a18b3e34f1961c1e0bc&scope.project.id=35a0f29fd4fe4e4fba533d39b8c6b932&role.id=02aba1f7cbdf4d6595e18af7fb5727e0 => generated 608 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:25:34.925119 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952b6120-8baf-4e3f-95a7-b156c744b9cb None None] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:25:34.925635 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952b6120-8baf-4e3f-95a7-b156c744b9cb None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:25:34.925746 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952b6120-8baf-4e3f-95a7-b156c744b9cb None None] PATH_INFO: `/` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:25:34.927040 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 558/1131] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jul 25 05:25:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:25:34.932853 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b262310c-87aa-4d6c-8ade-7b2b748f2392 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:25:34.933329 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b262310c-87aa-4d6c-8ade-7b2b748f2392 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:25:34.933618 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b262310c-87aa-4d6c-8ade-7b2b748f2392 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:25:34.950405 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-b262310c-87aa-4d6c-8ade-7b2b748f2392 None None] MFA Rules not processed for user `1eae540242d94c10b0ad33960a855f08`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:25:34.951583 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-b262310c-87aa-4d6c-8ade-7b2b748f2392 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:25:35.016541 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 574/1132] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jul 25 05:25:34 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 25 05:25:35.040050 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-952b6120-8baf-4e3f-95a7-b156c744b9cb None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:25:35.106700 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:25:35.138843 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-952b6120-8baf-4e3f-95a7-b156c744b9cb None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:25:35.172246 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-510b68ab-447d-4667-88a7-fcb20d54ba6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:25:35.173041 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510b68ab-447d-4667-88a7-fcb20d54ba6e None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:25:35.173321 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510b68ab-447d-4667-88a7-fcb20d54ba6e None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:25:35.173591 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510b68ab-447d-4667-88a7-fcb20d54ba6e None admin] PATH_INFO: `/v3/domains/default` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:25:35.174752 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-510b68ab-447d-4667-88a7-fcb20d54ba6e None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:25:35.176173 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-510b68ab-447d-4667-88a7-fcb20d54ba6e None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:25:35.177803 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 559/1133] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Jul 25 05:25:35 2022] GET /identity/v3/domains/default => generated 199 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:25:35.183613 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b262310c-87aa-4d6c-8ade-7b2b748f2392 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:25:35.185842 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:25:35.282340 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b262310c-87aa-4d6c-8ade-7b2b748f2392 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:25:35.315741 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b32ff4-56dd-4262-838c-3335e34cc85e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:25:35.316390 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b32ff4-56dd-4262-838c-3335e34cc85e None admin] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:25:35.316677 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b32ff4-56dd-4262-838c-3335e34cc85e None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:25:35.316986 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b32ff4-56dd-4262-838c-3335e34cc85e None admin] PATH_INFO: `/v3/projects` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:25:35.317708 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1b32ff4-56dd-4262-838c-3335e34cc85e None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:25:35.319146 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1b32ff4-56dd-4262-838c-3335e34cc85e None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:25:35.343672 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 575/1134] 10.222.0.33 () {60 vars in 1141 bytes} [Mon Jul 25 05:25:35 2022] POST /identity/v3/projects => generated 311 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 05:25:37.392309 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73c96b6-5a05-4c0b-a16b-46df8933cfc4 None None] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:25:37.392896 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73c96b6-5a05-4c0b-a16b-46df8933cfc4 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:25:37.393005 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73c96b6-5a05-4c0b-a16b-46df8933cfc4 None None] PATH_INFO: `/` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:25:37.394317 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 560/1135] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jul 25 05:25:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:25:37.399461 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5bf25fe-9427-4c70-b529-6bcce2a08034 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:25:37.399866 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5bf25fe-9427-4c70-b529-6bcce2a08034 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:25:37.400027 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5bf25fe-9427-4c70-b529-6bcce2a08034 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:25:37.417699 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-c5bf25fe-9427-4c70-b529-6bcce2a08034 None None] MFA Rules not processed for user `1eae540242d94c10b0ad33960a855f08`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:25:37.418909 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-c5bf25fe-9427-4c70-b529-6bcce2a08034 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:25:37.467955 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:25:37.486268 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 576/1136] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jul 25 05:25:37 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 25 05:25:37.510810 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73c96b6-5a05-4c0b-a16b-46df8933cfc4 None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:25:37.609860 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73c96b6-5a05-4c0b-a16b-46df8933cfc4 None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:25:37.642423 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e99b0335-eb6c-46b5-9bb3-47a3687b8eee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:25:37.643136 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99b0335-eb6c-46b5-9bb3-47a3687b8eee None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:25:37.643421 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99b0335-eb6c-46b5-9bb3-47a3687b8eee None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:25:37.643702 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99b0335-eb6c-46b5-9bb3-47a3687b8eee None admin] PATH_INFO: `/v3/domains/default` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:25:37.644906 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e99b0335-eb6c-46b5-9bb3-47a3687b8eee None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:25:37.646340 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e99b0335-eb6c-46b5-9bb3-47a3687b8eee None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:25:37.647939 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 561/1137] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Jul 25 05:25:37 2022] GET /identity/v3/domains/default => generated 199 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:25:37.653818 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5bf25fe-9427-4c70-b529-6bcce2a08034 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:25:37.749086 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5bf25fe-9427-4c70-b529-6bcce2a08034 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:25:37.781480 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc393524-fd88-469d-99b6-cfaf3621bb73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:25:37.782131 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc393524-fd88-469d-99b6-cfaf3621bb73 None admin] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:25:37.782416 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc393524-fd88-469d-99b6-cfaf3621bb73 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:25:37.782693 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc393524-fd88-469d-99b6-cfaf3621bb73 None admin] PATH_INFO: `/v3/users` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:25:37.783406 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc393524-fd88-469d-99b6-cfaf3621bb73 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:25:37.784840 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc393524-fd88-469d-99b6-cfaf3621bb73 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:25:37.807980 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 577/1138] 10.222.0.33 () {60 vars in 1132 bytes} [Mon Jul 25 05:25:37 2022] POST /identity/v3/users => generated 286 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 05:25:39.846960 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a72fae-a4c3-4eb6-829c-36390c6dd516 None None] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:25:39.847463 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a72fae-a4c3-4eb6-829c-36390c6dd516 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:25:39.847565 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a72fae-a4c3-4eb6-829c-36390c6dd516 None None] PATH_INFO: `/` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:25:39.848894 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 562/1139] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jul 25 05:25:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:25:39.855083 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bec33f2-140c-4af3-bd76-a3d283f3c675 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:25:39.855504 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bec33f2-140c-4af3-bd76-a3d283f3c675 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:25:39.855644 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bec33f2-140c-4af3-bd76-a3d283f3c675 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:25:39.872025 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-6bec33f2-140c-4af3-bd76-a3d283f3c675 None None] MFA Rules not processed for user `1eae540242d94c10b0ad33960a855f08`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:25:39.873252 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-6bec33f2-140c-4af3-bd76-a3d283f3c675 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:25:39.907485 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:25:39.940729 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 578/1140] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jul 25 05:25:39 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 25 05:25:39.964970 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a72fae-a4c3-4eb6-829c-36390c6dd516 None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:25:39.968718 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:25:40.063254 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a72fae-a4c3-4eb6-829c-36390c6dd516 None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:25:40.097332 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a0cad3-69bb-4fbd-a1c8-49b691f5640e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:25:40.098105 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a0cad3-69bb-4fbd-a1c8-49b691f5640e None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:25:40.098386 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a0cad3-69bb-4fbd-a1c8-49b691f5640e None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:25:40.098662 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a0cad3-69bb-4fbd-a1c8-49b691f5640e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:25:40.108645 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05a0cad3-69bb-4fbd-a1c8-49b691f5640e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 05:25:40.116087 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application [None req-05a0cad3-69bb-4fbd-a1c8-49b691f5640e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 05:25:40.116087 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:25:40.116087 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:25:40.116087 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:25:40.116087 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:25:40.116087 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:25:40.116087 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:25:40.116087 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:25:40.116087 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:25:40.116087 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:25:40.116087 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:25:40.116087 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:25:40.116087 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 05:25:40.116087 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 05:25:40.116087 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 05:25:40.116087 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 05:25:40.117322 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 05:25:40.117322 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 05:25:40.117322 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 05:25:40.117322 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 05:25:40.117322 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:25:40.117322 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:25:40.117322 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:25:40.117322 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:25:40.117322 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:25:40.117322 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:25:40.117322 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:25:40.117322 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application with Lock( Jul 25 05:25:40.117322 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:25:40.117322 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:25:40.117322 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:25:40.117322 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:25:40.118300 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:25:40.118300 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:25:40.118300 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:25:40.118300 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:25:40.118300 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 05:25:40.118300 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 05:25:40.118300 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 05:25:40.118300 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 05:25:40.118300 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 05:25:40.118300 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 05:25:40.118300 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 05:25:40.118300 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Jul 25 05:25:40.118300 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 563/1141] 10.222.0.33 () {56 vars in 1096 bytes} [Mon Jul 25 05:25:39 2022] GET /identity/v3/roles/admin => generated 83 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:25:40.123326 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bec33f2-140c-4af3-bd76-a3d283f3c675 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:25:40.221655 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bec33f2-140c-4af3-bd76-a3d283f3c675 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:25:40.254643 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c77470-8521-406b-b9c1-986b5824e320 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:25:40.255303 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c77470-8521-406b-b9c1-986b5824e320 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:25:40.255587 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c77470-8521-406b-b9c1-986b5824e320 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:25:40.255867 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c77470-8521-406b-b9c1-986b5824e320 None admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:25:40.256616 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18c77470-8521-406b-b9c1-986b5824e320 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:25:40.257005 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18c77470-8521-406b-b9c1-986b5824e320 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:25:40.258373 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18c77470-8521-406b-b9c1-986b5824e320 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:25:40.267142 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 579/1142] 10.222.0.33 () {56 vars in 1099 bytes} [Mon Jul 25 05:25:40 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:25:40.272930 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a0cad3-69bb-4fbd-a1c8-49b691f5640e None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:25:40.334881 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:25:40.371374 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a0cad3-69bb-4fbd-a1c8-49b691f5640e None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:25:40.403534 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-963a20e8-b280-4ef4-a5fb-c55b292edd74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:25:40.404389 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963a20e8-b280-4ef4-a5fb-c55b292edd74 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:25:40.404673 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963a20e8-b280-4ef4-a5fb-c55b292edd74 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:25:40.404984 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963a20e8-b280-4ef4-a5fb-c55b292edd74 None admin] PATH_INFO: `/v3/users/aa00e549a3d44f39b8502300ace7c7b8` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:25:40.417699 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-963a20e8-b280-4ef4-a5fb-c55b292edd74 None admin] RBAC: Authorizing `identity:get_user(user_id=aa00e549a3d44f39b8502300ace7c7b8)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:25:40.419137 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-963a20e8-b280-4ef4-a5fb-c55b292edd74 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:25:40.420762 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 564/1143] 10.222.0.33 () {56 vars in 1177 bytes} [Mon Jul 25 05:25:40 2022] GET /identity/v3/users/aa00e549a3d44f39b8502300ace7c7b8 => generated 286 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:25:40.426480 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c77470-8521-406b-b9c1-986b5824e320 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:25:40.458209 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:25:40.525214 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c77470-8521-406b-b9c1-986b5824e320 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:25:40.559437 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f01c7edc-8fd5-4640-a878-7f5556c60c70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:25:40.560084 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01c7edc-8fd5-4640-a878-7f5556c60c70 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:25:40.560374 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01c7edc-8fd5-4640-a878-7f5556c60c70 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:25:40.560657 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01c7edc-8fd5-4640-a878-7f5556c60c70 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:25:40.627057 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f01c7edc-8fd5-4640-a878-7f5556c60c70 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:25:40.628441 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f01c7edc-8fd5-4640-a878-7f5556c60c70 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:25:40.656314 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:25:40.723047 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 580/1144] 10.222.0.33 () {58 vars in 1282 bytes} [Mon Jul 25 05:25:40 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 297 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 25 05:25:40.728899 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-963a20e8-b280-4ef4-a5fb-c55b292edd74 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:25:40.827312 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-963a20e8-b280-4ef4-a5fb-c55b292edd74 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:25:40.849601 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:25:40.861137 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a49d22-8866-454c-932d-1f215eddf126 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:25:40.861993 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a49d22-8866-454c-932d-1f215eddf126 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:25:40.862280 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a49d22-8866-454c-932d-1f215eddf126 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:25:40.862557 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a49d22-8866-454c-932d-1f215eddf126 None admin] PATH_INFO: `/v3/projects/5fa9e7663bf04a3fbf6e83f03c02871c` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:25:40.863731 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27a49d22-8866-454c-932d-1f215eddf126 None admin] RBAC: Authorizing `identity:get_project(project_id=5fa9e7663bf04a3fbf6e83f03c02871c)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:25:40.865194 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27a49d22-8866-454c-932d-1f215eddf126 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:25:40.866891 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 565/1145] 10.222.0.33 () {56 vars in 1186 bytes} [Mon Jul 25 05:25:40 2022] GET /identity/v3/projects/5fa9e7663bf04a3fbf6e83f03c02871c => generated 311 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:25:40.873125 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f01c7edc-8fd5-4640-a878-7f5556c60c70 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:25:40.972261 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f01c7edc-8fd5-4640-a878-7f5556c60c70 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:25:41.005922 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:25:41.012298 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-543d25e7-d5cc-4f63-b052-77edb3a6c550 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:25:41.012988 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543d25e7-d5cc-4f63-b052-77edb3a6c550 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:25:41.013275 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543d25e7-d5cc-4f63-b052-77edb3a6c550 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:25:41.013551 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543d25e7-d5cc-4f63-b052-77edb3a6c550 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:25:41.014390 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-543d25e7-d5cc-4f63-b052-77edb3a6c550 None admin] RBAC: Adding query filter params (role.id=39e5c666285a42fe8fc2c3afca6260d1, scope.project.id=5fa9e7663bf04a3fbf6e83f03c02871c, user.id=aa00e549a3d44f39b8502300ace7c7b8) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:25:41.014702 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-543d25e7-d5cc-4f63-b052-77edb3a6c550 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:25:41.016124 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-543d25e7-d5cc-4f63-b052-77edb3a6c550 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:25:41.021538 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 581/1146] 10.222.0.33 () {56 vars in 1374 bytes} [Mon Jul 25 05:25:40 2022] GET /identity/v3/role_assignments?user.id=aa00e549a3d44f39b8502300ace7c7b8&scope.project.id=5fa9e7663bf04a3fbf6e83f03c02871c&role.id=39e5c666285a42fe8fc2c3afca6260d1 => generated 259 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:25:42.996422 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f89919-6fe6-4027-82b8-807478dbde58 None None] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:25:42.996987 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f89919-6fe6-4027-82b8-807478dbde58 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:25:42.996987 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f89919-6fe6-4027-82b8-807478dbde58 None None] PATH_INFO: `/` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:25:42.998377 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 566/1147] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jul 25 05:25:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:25:43.003710 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dee97de-4df7-436a-b350-0693a963433a None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:25:43.004115 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dee97de-4df7-436a-b350-0693a963433a None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:25:43.004266 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dee97de-4df7-436a-b350-0693a963433a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:25:43.021297 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-6dee97de-4df7-436a-b350-0693a963433a None None] MFA Rules not processed for user `1eae540242d94c10b0ad33960a855f08`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:25:43.022472 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-6dee97de-4df7-436a-b350-0693a963433a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:25:43.086565 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 582/1148] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jul 25 05:25:43 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 25 05:25:43.109301 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f89919-6fe6-4027-82b8-807478dbde58 None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:25:43.206922 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f89919-6fe6-4027-82b8-807478dbde58 None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:25:43.240216 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e820a5b-f8f5-431e-9479-8044eb5a9a1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:25:43.241029 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e820a5b-f8f5-431e-9479-8044eb5a9a1d None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:25:43.241316 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e820a5b-f8f5-431e-9479-8044eb5a9a1d None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:25:43.241612 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e820a5b-f8f5-431e-9479-8044eb5a9a1d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:25:43.252494 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e820a5b-f8f5-431e-9479-8044eb5a9a1d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 05:25:43.261281 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application [None req-6e820a5b-f8f5-431e-9479-8044eb5a9a1d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 05:25:43.261281 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:25:43.261281 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:25:43.261281 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:25:43.261281 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:25:43.261281 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:25:43.261281 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:25:43.261281 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:25:43.261281 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:25:43.261281 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:25:43.261281 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:25:43.261281 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:25:43.261281 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 05:25:43.261281 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 05:25:43.261281 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 05:25:43.261281 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 05:25:43.262427 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 05:25:43.262427 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 05:25:43.262427 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 05:25:43.262427 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 05:25:43.262427 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:25:43.262427 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:25:43.262427 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:25:43.262427 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:25:43.262427 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:25:43.262427 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:25:43.262427 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:25:43.262427 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application with Lock( Jul 25 05:25:43.262427 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:25:43.262427 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:25:43.262427 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:25:43.262427 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:25:43.268636 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:25:43.268636 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:25:43.268636 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:25:43.268636 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:25:43.268636 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 05:25:43.268636 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 05:25:43.268636 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 05:25:43.268636 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 05:25:43.268636 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 05:25:43.268636 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 05:25:43.268636 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 05:25:43.268636 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Jul 25 05:25:43.268636 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 567/1149] 10.222.0.33 () {56 vars in 1096 bytes} [Mon Jul 25 05:25:43 2022] GET /identity/v3/roles/admin => generated 83 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:25:43.269579 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dee97de-4df7-436a-b350-0693a963433a None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:25:43.362791 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dee97de-4df7-436a-b350-0693a963433a None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:25:43.388047 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:25:43.400640 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d58d4285-8137-4672-b171-0e140fca4f73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:25:43.401348 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58d4285-8137-4672-b171-0e140fca4f73 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:25:43.401631 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58d4285-8137-4672-b171-0e140fca4f73 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:25:43.401908 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58d4285-8137-4672-b171-0e140fca4f73 None admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:25:43.402662 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d58d4285-8137-4672-b171-0e140fca4f73 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:25:43.402970 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d58d4285-8137-4672-b171-0e140fca4f73 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:25:43.404330 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d58d4285-8137-4672-b171-0e140fca4f73 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:25:43.412495 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 583/1150] 10.222.0.33 () {56 vars in 1099 bytes} [Mon Jul 25 05:25:43 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 05:25:43.418754 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e820a5b-f8f5-431e-9479-8044eb5a9a1d None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:25:43.468425 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:25:43.516501 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e820a5b-f8f5-431e-9479-8044eb5a9a1d None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:25:43.547730 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d030c2-0ce7-4a77-8ada-77afad8b0869 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:25:43.548685 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d030c2-0ce7-4a77-8ada-77afad8b0869 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:25:43.549094 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d030c2-0ce7-4a77-8ada-77afad8b0869 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:25:43.549371 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d030c2-0ce7-4a77-8ada-77afad8b0869 None admin] PATH_INFO: `/v3/users/aa00e549a3d44f39b8502300ace7c7b8` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:25:43.550668 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7d030c2-0ce7-4a77-8ada-77afad8b0869 None admin] RBAC: Authorizing `identity:get_user(user_id=aa00e549a3d44f39b8502300ace7c7b8)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:25:43.552093 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7d030c2-0ce7-4a77-8ada-77afad8b0869 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:25:43.553751 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 568/1151] 10.222.0.33 () {56 vars in 1177 bytes} [Mon Jul 25 05:25:43 2022] GET /identity/v3/users/aa00e549a3d44f39b8502300ace7c7b8 => generated 286 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:25:43.560296 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d58d4285-8137-4672-b171-0e140fca4f73 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:25:43.657712 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d58d4285-8137-4672-b171-0e140fca4f73 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:25:43.690775 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a4a02f-a61b-4f8a-9a09-02cf52579fae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:25:43.691622 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a4a02f-a61b-4f8a-9a09-02cf52579fae None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:25:43.691910 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a4a02f-a61b-4f8a-9a09-02cf52579fae None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:25:43.692194 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a4a02f-a61b-4f8a-9a09-02cf52579fae None admin] PATH_INFO: `/v3/projects/5fa9e7663bf04a3fbf6e83f03c02871c` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:25:43.693403 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45a4a02f-a61b-4f8a-9a09-02cf52579fae None admin] RBAC: Authorizing `identity:get_project(project_id=5fa9e7663bf04a3fbf6e83f03c02871c)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:25:43.694841 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45a4a02f-a61b-4f8a-9a09-02cf52579fae None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:25:43.696523 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 584/1152] 10.222.0.33 () {56 vars in 1186 bytes} [Mon Jul 25 05:25:43 2022] GET /identity/v3/projects/5fa9e7663bf04a3fbf6e83f03c02871c => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:25:43.702415 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d030c2-0ce7-4a77-8ada-77afad8b0869 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:25:43.799283 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d030c2-0ce7-4a77-8ada-77afad8b0869 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:25:43.816898 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:25:43.833635 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c69a7a14-3d10-47e1-9949-f7153933bf01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:25:43.834524 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69a7a14-3d10-47e1-9949-f7153933bf01 None admin] REQUEST_METHOD: `PUT` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:25:43.834811 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69a7a14-3d10-47e1-9949-f7153933bf01 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:25:43.835089 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69a7a14-3d10-47e1-9949-f7153933bf01 None admin] PATH_INFO: `/v3/projects/5fa9e7663bf04a3fbf6e83f03c02871c/users/aa00e549a3d44f39b8502300ace7c7b8/roles/39e5c666285a42fe8fc2c3afca6260d1` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:25:43.837076 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c69a7a14-3d10-47e1-9949-f7153933bf01 None admin] RBAC: Authorizing `identity:create_grant(project_id=5fa9e7663bf04a3fbf6e83f03c02871c, user_id=aa00e549a3d44f39b8502300ace7c7b8, role_id=39e5c666285a42fe8fc2c3afca6260d1)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:25:43.838543 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c69a7a14-3d10-47e1-9949-f7153933bf01 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:25:43.850297 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 569/1153] 10.222.0.33 () {58 vars in 1439 bytes} [Mon Jul 25 05:25:43 2022] PUT /identity/v3/projects/5fa9e7663bf04a3fbf6e83f03c02871c/users/aa00e549a3d44f39b8502300ace7c7b8/roles/39e5c666285a42fe8fc2c3afca6260d1 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 05:25:45.832303 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d422e130-0756-4487-88ce-50d334ed4cdf None None] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:25:45.832817 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d422e130-0756-4487-88ce-50d334ed4cdf None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:25:45.832913 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d422e130-0756-4487-88ce-50d334ed4cdf None None] PATH_INFO: `/` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:25:45.834258 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 585/1154] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jul 25 05:25:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:25:45.839481 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d1855e-1e86-47c1-a923-4389d5c57b3f None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:25:45.839915 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d1855e-1e86-47c1-a923-4389d5c57b3f None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:25:45.840142 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d1855e-1e86-47c1-a923-4389d5c57b3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:25:45.857026 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-83d1855e-1e86-47c1-a923-4389d5c57b3f None None] MFA Rules not processed for user `1eae540242d94c10b0ad33960a855f08`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:25:45.858197 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-83d1855e-1e86-47c1-a923-4389d5c57b3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:25:45.921525 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 570/1155] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jul 25 05:25:45 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 25 05:25:45.944993 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d422e130-0756-4487-88ce-50d334ed4cdf None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:25:46.000652 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:25:46.044848 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d422e130-0756-4487-88ce-50d334ed4cdf None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:25:46.076544 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83538f67-03fd-479f-8dbc-fde1494007b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:25:46.077287 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83538f67-03fd-479f-8dbc-fde1494007b3 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:25:46.077570 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83538f67-03fd-479f-8dbc-fde1494007b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:25:46.077844 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83538f67-03fd-479f-8dbc-fde1494007b3 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:25:46.087485 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83538f67-03fd-479f-8dbc-fde1494007b3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 05:25:46.095607 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application [None req-83538f67-03fd-479f-8dbc-fde1494007b3 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 05:25:46.095607 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:25:46.095607 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:25:46.095607 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:25:46.095607 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:25:46.095607 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:25:46.095607 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:25:46.095607 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:25:46.095607 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:25:46.095607 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:25:46.095607 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:25:46.095607 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:25:46.095607 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 05:25:46.095607 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 05:25:46.095607 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 05:25:46.095607 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 05:25:46.096745 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 05:25:46.096745 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 05:25:46.096745 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 05:25:46.096745 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 05:25:46.096745 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:25:46.096745 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:25:46.096745 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:25:46.096745 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:25:46.096745 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:25:46.096745 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:25:46.096745 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:25:46.096745 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application with Lock( Jul 25 05:25:46.096745 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:25:46.096745 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:25:46.096745 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:25:46.096745 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:25:46.097991 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:25:46.097991 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:25:46.097991 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:25:46.097991 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:25:46.097991 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 05:25:46.097991 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 05:25:46.097991 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 05:25:46.097991 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 05:25:46.097991 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 05:25:46.097991 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 05:25:46.097991 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 05:25:46.097991 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Jul 25 05:25:46.097991 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 586/1156] 10.222.0.33 () {56 vars in 1096 bytes} [Mon Jul 25 05:25:45 2022] GET /identity/v3/roles/admin => generated 83 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:25:46.102645 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d1855e-1e86-47c1-a923-4389d5c57b3f None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:25:46.199533 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d1855e-1e86-47c1-a923-4389d5c57b3f None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:25:46.208465 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:25:46.234100 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-015f029b-d37d-4523-a80f-b4da761cd29a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:25:46.234763 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015f029b-d37d-4523-a80f-b4da761cd29a None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:25:46.235050 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015f029b-d37d-4523-a80f-b4da761cd29a None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:25:46.235331 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015f029b-d37d-4523-a80f-b4da761cd29a None admin] PATH_INFO: `/v3/roles` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:25:46.236079 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-015f029b-d37d-4523-a80f-b4da761cd29a None admin] RBAC: Adding query filter params (name=admin) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:25:46.236391 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-015f029b-d37d-4523-a80f-b4da761cd29a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:25:46.237805 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-015f029b-d37d-4523-a80f-b4da761cd29a None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:25:46.246596 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 571/1157] 10.222.0.33 () {56 vars in 1099 bytes} [Mon Jul 25 05:25:46 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:25:46.252364 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83538f67-03fd-479f-8dbc-fde1494007b3 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:25:46.354507 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83538f67-03fd-479f-8dbc-fde1494007b3 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:25:46.370253 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:25:46.387546 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6eadbbf-6022-4cc1-8b54-9dd05391865a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:25:46.388398 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6eadbbf-6022-4cc1-8b54-9dd05391865a None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:25:46.388677 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6eadbbf-6022-4cc1-8b54-9dd05391865a None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:25:46.388978 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6eadbbf-6022-4cc1-8b54-9dd05391865a None admin] PATH_INFO: `/v3/users/aa00e549a3d44f39b8502300ace7c7b8` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:25:46.390144 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6eadbbf-6022-4cc1-8b54-9dd05391865a None admin] RBAC: Authorizing `identity:get_user(user_id=aa00e549a3d44f39b8502300ace7c7b8)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:25:46.391558 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6eadbbf-6022-4cc1-8b54-9dd05391865a None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:25:46.393213 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 587/1158] 10.222.0.33 () {56 vars in 1177 bytes} [Mon Jul 25 05:25:46 2022] GET /identity/v3/users/aa00e549a3d44f39b8502300ace7c7b8 => generated 286 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:25:46.399080 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-015f029b-d37d-4523-a80f-b4da761cd29a None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:25:46.497409 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-015f029b-d37d-4523-a80f-b4da761cd29a None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:25:46.529918 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac66245-5860-4a55-a37d-8cb05840c456 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:25:46.530552 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac66245-5860-4a55-a37d-8cb05840c456 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:25:46.530838 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac66245-5860-4a55-a37d-8cb05840c456 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:25:46.531117 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac66245-5860-4a55-a37d-8cb05840c456 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:25:46.569912 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:25:46.598188 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ac66245-5860-4a55-a37d-8cb05840c456 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:25:46.599625 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ac66245-5860-4a55-a37d-8cb05840c456 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:25:46.694955 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 572/1159] 10.222.0.33 () {58 vars in 1282 bytes} [Mon Jul 25 05:25:46 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 296 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 25 05:25:46.701663 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6eadbbf-6022-4cc1-8b54-9dd05391865a None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:25:46.796335 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6eadbbf-6022-4cc1-8b54-9dd05391865a None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:25:46.831702 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3fb9aeb-681b-430f-99e2-fd9eab45ac4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:25:46.832558 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3fb9aeb-681b-430f-99e2-fd9eab45ac4b None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:25:46.833107 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3fb9aeb-681b-430f-99e2-fd9eab45ac4b None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:25:46.833221 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3fb9aeb-681b-430f-99e2-fd9eab45ac4b None admin] PATH_INFO: `/v3/projects/5fa9e7663bf04a3fbf6e83f03c02871c` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:25:46.834314 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3fb9aeb-681b-430f-99e2-fd9eab45ac4b None admin] RBAC: Authorizing `identity:get_project(project_id=5fa9e7663bf04a3fbf6e83f03c02871c)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:25:46.835747 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3fb9aeb-681b-430f-99e2-fd9eab45ac4b None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:25:46.837484 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 588/1160] 10.222.0.33 () {56 vars in 1186 bytes} [Mon Jul 25 05:25:46 2022] GET /identity/v3/projects/5fa9e7663bf04a3fbf6e83f03c02871c => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:25:46.843605 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac66245-5860-4a55-a37d-8cb05840c456 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:25:46.908800 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:25:46.938188 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac66245-5860-4a55-a37d-8cb05840c456 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:25:46.972073 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab1eac99-f7fd-4d61-93bd-bd2bdeb3809d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:25:46.972648 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1eac99-f7fd-4d61-93bd-bd2bdeb3809d None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:25:46.972960 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1eac99-f7fd-4d61-93bd-bd2bdeb3809d None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:25:46.973238 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1eac99-f7fd-4d61-93bd-bd2bdeb3809d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:25:46.974065 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab1eac99-f7fd-4d61-93bd-bd2bdeb3809d None admin] RBAC: Adding query filter params (role.id=39e5c666285a42fe8fc2c3afca6260d1, scope.project.id=5fa9e7663bf04a3fbf6e83f03c02871c, user.id=aa00e549a3d44f39b8502300ace7c7b8) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:25:46.974378 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab1eac99-f7fd-4d61-93bd-bd2bdeb3809d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:25:46.975794 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab1eac99-f7fd-4d61-93bd-bd2bdeb3809d None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:25:46.982277 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 573/1161] 10.222.0.33 () {56 vars in 1374 bytes} [Mon Jul 25 05:25:46 2022] GET /identity/v3/role_assignments?user.id=aa00e549a3d44f39b8502300ace7c7b8&scope.project.id=5fa9e7663bf04a3fbf6e83f03c02871c&role.id=39e5c666285a42fe8fc2c3afca6260d1 => generated 608 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:25:49.139850 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086373e2-c72b-464d-a084-1d0d63c0e5b6 None None] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:25:49.140379 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086373e2-c72b-464d-a084-1d0d63c0e5b6 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:25:49.140379 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086373e2-c72b-464d-a084-1d0d63c0e5b6 None None] PATH_INFO: `/` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:25:49.141733 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 589/1162] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jul 25 05:25:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:25:49.146852 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90adaad4-fc7c-4c24-a907-e491aad49633 None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:25:49.147285 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90adaad4-fc7c-4c24-a907-e491aad49633 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:25:49.147390 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90adaad4-fc7c-4c24-a907-e491aad49633 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:25:49.164095 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-90adaad4-fc7c-4c24-a907-e491aad49633 None None] MFA Rules not processed for user `1eae540242d94c10b0ad33960a855f08`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:25:49.165303 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-90adaad4-fc7c-4c24-a907-e491aad49633 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:25:49.230538 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 574/1163] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jul 25 05:25:49 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 25 05:25:49.252474 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-086373e2-c72b-464d-a084-1d0d63c0e5b6 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:25:49.305308 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:25:49.352481 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-086373e2-c72b-464d-a084-1d0d63c0e5b6 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:25:49.384644 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c77dd8e1-df0f-4e73-ba3d-2548423bc4a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:25:49.385970 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77dd8e1-df0f-4e73-ba3d-2548423bc4a6 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:25:49.386274 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77dd8e1-df0f-4e73-ba3d-2548423bc4a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:25:49.386557 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77dd8e1-df0f-4e73-ba3d-2548423bc4a6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:25:49.451487 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c77dd8e1-df0f-4e73-ba3d-2548423bc4a6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:25:49.452897 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c77dd8e1-df0f-4e73-ba3d-2548423bc4a6 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:25:49.501473 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:25:49.549785 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 590/1164] 10.222.0.33 () {58 vars in 1282 bytes} [Mon Jul 25 05:25:49 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 298 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 25 05:25:49.555878 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90adaad4-fc7c-4c24-a907-e491aad49633 None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:25:49.616305 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:25:49.653573 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90adaad4-fc7c-4c24-a907-e491aad49633 None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:25:49.686091 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df5f31c6-e606-41d5-9d14-f59b6f52627a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:25:49.686877 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5f31c6-e606-41d5-9d14-f59b6f52627a None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:25:49.687164 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5f31c6-e606-41d5-9d14-f59b6f52627a None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:25:49.687444 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5f31c6-e606-41d5-9d14-f59b6f52627a None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:25:49.694233 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df5f31c6-e606-41d5-9d14-f59b6f52627a None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:25:49.695668 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df5f31c6-e606-41d5-9d14-f59b6f52627a None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:25:49.710164 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application [None req-df5f31c6-e606-41d5-9d14-f59b6f52627a None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Jul 25 05:25:49.710164 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:25:49.710164 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Jul 25 05:25:49.710164 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Jul 25 05:25:49.710164 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 25 05:25:49.710164 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 25 05:25:49.710164 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 25 05:25:49.710164 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 25 05:25:49.710164 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Jul 25 05:25:49.710164 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Jul 25 05:25:49.710164 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 25 05:25:49.710164 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Jul 25 05:25:49.710164 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:25:49.710164 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:25:49.710164 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:25:49.710164 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:25:49.710164 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:25:49.711354 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:25:49.711354 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:25:49.711354 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:25:49.711354 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:25:49.711354 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:25:49.711354 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:25:49.711354 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 05:25:49.711354 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 05:25:49.711354 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 05:25:49.711354 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 05:25:49.711354 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:25:49.711354 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:25:49.711354 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:25:49.711354 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:25:49.711354 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:25:49.711354 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:25:49.712226 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:25:49.712226 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application with Lock( Jul 25 05:25:49.712226 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:25:49.712226 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:25:49.712226 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:25:49.712226 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:25:49.712226 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:25:49.712226 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:25:49.712226 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:25:49.712226 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:25:49.712226 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Jul 25 05:25:49.712226 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 05:25:49.712226 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Jul 25 05:25:49.712226 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Jul 25 05:25:49.712226 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 575/1165] 10.222.0.33 () {56 vars in 1117 bytes} [Mon Jul 25 05:25:49 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:25:49.717546 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c77dd8e1-df0f-4e73-ba3d-2548423bc4a6 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:25:49.811578 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c77dd8e1-df0f-4e73-ba3d-2548423bc4a6 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:25:49.844158 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ecb054d-a962-4cea-8c5e-a9f4bc4faaa7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:25:49.844829 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ecb054d-a962-4cea-8c5e-a9f4bc4faaa7 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:25:49.845116 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ecb054d-a962-4cea-8c5e-a9f4bc4faaa7 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:25:49.845393 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ecb054d-a962-4cea-8c5e-a9f4bc4faaa7 None admin] PATH_INFO: `/v3/domains` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:25:49.846108 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ecb054d-a962-4cea-8c5e-a9f4bc4faaa7 None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:25:49.846416 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ecb054d-a962-4cea-8c5e-a9f4bc4faaa7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:25:49.847772 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ecb054d-a962-4cea-8c5e-a9f4bc4faaa7 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:25:49.854736 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 591/1166] 10.222.0.33 () {56 vars in 1115 bytes} [Mon Jul 25 05:25:49 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:25:51.880638 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7bf547-64e4-4582-a6d3-2c8300fd671e None None] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:25:51.881179 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7bf547-64e4-4582-a6d3-2c8300fd671e None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:25:51.881297 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7bf547-64e4-4582-a6d3-2c8300fd671e None None] PATH_INFO: `/` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:25:51.882608 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 576/1167] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jul 25 05:25:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:25:51.888283 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3550aa7-0187-4308-98d9-feacf6411e5a None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:25:51.888665 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3550aa7-0187-4308-98d9-feacf6411e5a None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:25:51.888884 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3550aa7-0187-4308-98d9-feacf6411e5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:25:51.905955 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-e3550aa7-0187-4308-98d9-feacf6411e5a None None] MFA Rules not processed for user `1eae540242d94c10b0ad33960a855f08`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:25:51.907135 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-e3550aa7-0187-4308-98d9-feacf6411e5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:25:51.917674 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:25:51.972671 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 592/1168] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jul 25 05:25:51 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 25 05:25:51.995957 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc7bf547-64e4-4582-a6d3-2c8300fd671e None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:25:52.095682 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc7bf547-64e4-4582-a6d3-2c8300fd671e None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:25:52.117480 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:25:52.128492 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5415473c-539c-4e16-868c-adf622d976bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:25:52.129173 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5415473c-539c-4e16-868c-adf622d976bb None admin] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:25:52.129463 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5415473c-539c-4e16-868c-adf622d976bb None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:25:52.129743 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5415473c-539c-4e16-868c-adf622d976bb None admin] PATH_INFO: `/v3/domains` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:25:52.130485 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5415473c-539c-4e16-868c-adf622d976bb None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:25:52.131851 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5415473c-539c-4e16-868c-adf622d976bb None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:25:52.145556 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 577/1169] 10.222.0.33 () {60 vars in 1138 bytes} [Mon Jul 25 05:25:51 2022] POST /identity/v3/domains => generated 267 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 05:25:54.191535 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5896b3d1-cf26-4168-b94e-f012b4eec93f None None] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:25:54.192065 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5896b3d1-cf26-4168-b94e-f012b4eec93f None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:25:54.192065 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5896b3d1-cf26-4168-b94e-f012b4eec93f None None] PATH_INFO: `/` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:25:54.193421 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 593/1170] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jul 25 05:25:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:25:54.199397 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285751ef-7c86-43df-ab61-4149f3caba7c None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:25:54.199812 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285751ef-7c86-43df-ab61-4149f3caba7c None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:25:54.199968 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285751ef-7c86-43df-ab61-4149f3caba7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:25:54.216529 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-285751ef-7c86-43df-ab61-4149f3caba7c None None] MFA Rules not processed for user `1eae540242d94c10b0ad33960a855f08`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:25:54.217766 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-285751ef-7c86-43df-ab61-4149f3caba7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:25:54.287201 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 578/1171] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jul 25 05:25:54 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 25 05:25:54.311073 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5896b3d1-cf26-4168-b94e-f012b4eec93f None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:25:54.410560 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5896b3d1-cf26-4168-b94e-f012b4eec93f None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:25:54.443675 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d31ac2-ff66-4732-bc20-9b5956422bd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:25:54.444463 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d31ac2-ff66-4732-bc20-9b5956422bd0 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:25:54.444751 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d31ac2-ff66-4732-bc20-9b5956422bd0 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:25:54.445065 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d31ac2-ff66-4732-bc20-9b5956422bd0 None admin] PATH_INFO: `/v3/domains/a365ae9b0f1946bf8940d700925f67a1` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:25:54.455475 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28d31ac2-ff66-4732-bc20-9b5956422bd0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=a365ae9b0f1946bf8940d700925f67a1)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:25:54.456951 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28d31ac2-ff66-4732-bc20-9b5956422bd0 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:25:54.458580 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 594/1172] 10.222.0.33 () {56 vars in 1183 bytes} [Mon Jul 25 05:25:54 2022] GET /identity/v3/domains/a365ae9b0f1946bf8940d700925f67a1 => generated 267 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:25:54.465071 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-285751ef-7c86-43df-ab61-4149f3caba7c None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:25:54.559655 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-285751ef-7c86-43df-ab61-4149f3caba7c None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:25:54.573702 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:25:54.594176 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03be78ab-284d-4129-b3d2-3378550e3ff9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:25:54.594829 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03be78ab-284d-4129-b3d2-3378550e3ff9 None admin] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:25:54.595117 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03be78ab-284d-4129-b3d2-3378550e3ff9 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:25:54.595394 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03be78ab-284d-4129-b3d2-3378550e3ff9 None admin] PATH_INFO: `/v3/projects` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:25:54.596112 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03be78ab-284d-4129-b3d2-3378550e3ff9 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:25:54.597563 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03be78ab-284d-4129-b3d2-3378550e3ff9 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:25:54.621880 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 579/1173] 10.222.0.33 () {60 vars in 1141 bytes} [Mon Jul 25 05:25:54 2022] POST /identity/v3/projects => generated 361 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 05:25:56.654288 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88cca54c-3900-4fb1-81f6-a1a0ed4a6450 None None] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:25:56.654865 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88cca54c-3900-4fb1-81f6-a1a0ed4a6450 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:25:56.654865 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88cca54c-3900-4fb1-81f6-a1a0ed4a6450 None None] PATH_INFO: `/` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:25:56.656228 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 595/1174] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jul 25 05:25:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:25:56.661367 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a095a3-f9bc-45ee-9c6a-86445ea858a0 None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:25:56.661753 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a095a3-f9bc-45ee-9c6a-86445ea858a0 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:25:56.661925 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a095a3-f9bc-45ee-9c6a-86445ea858a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:25:56.678787 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-74a095a3-f9bc-45ee-9c6a-86445ea858a0 None None] MFA Rules not processed for user `1eae540242d94c10b0ad33960a855f08`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:25:56.679959 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-74a095a3-f9bc-45ee-9c6a-86445ea858a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:25:56.743675 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 580/1175] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jul 25 05:25:56 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 25 05:25:56.768271 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88cca54c-3900-4fb1-81f6-a1a0ed4a6450 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:25:56.777620 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:25:56.867388 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88cca54c-3900-4fb1-81f6-a1a0ed4a6450 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:25:56.899475 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c7bb1b7-bd0e-4e30-a830-ab2d2b5545a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:25:56.900319 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7bb1b7-bd0e-4e30-a830-ab2d2b5545a5 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:25:56.900631 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7bb1b7-bd0e-4e30-a830-ab2d2b5545a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:25:56.900927 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7bb1b7-bd0e-4e30-a830-ab2d2b5545a5 None admin] PATH_INFO: `/v3/domains/a365ae9b0f1946bf8940d700925f67a1` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:25:56.902189 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c7bb1b7-bd0e-4e30-a830-ab2d2b5545a5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=a365ae9b0f1946bf8940d700925f67a1)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:25:56.903598 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c7bb1b7-bd0e-4e30-a830-ab2d2b5545a5 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:25:56.905274 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 596/1176] 10.222.0.33 () {56 vars in 1183 bytes} [Mon Jul 25 05:25:56 2022] GET /identity/v3/domains/a365ae9b0f1946bf8940d700925f67a1 => generated 267 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:25:56.910914 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a095a3-f9bc-45ee-9c6a-86445ea858a0 None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:25:57.000768 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:25:57.006917 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a095a3-f9bc-45ee-9c6a-86445ea858a0 None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:25:57.040647 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff6ae40c-9953-4587-962a-87bc9a31cc76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:25:57.041833 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6ae40c-9953-4587-962a-87bc9a31cc76 None admin] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:25:57.042119 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6ae40c-9953-4587-962a-87bc9a31cc76 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:25:57.042395 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6ae40c-9953-4587-962a-87bc9a31cc76 None admin] PATH_INFO: `/v3/users` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:25:57.043102 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff6ae40c-9953-4587-962a-87bc9a31cc76 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:25:57.044529 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff6ae40c-9953-4587-962a-87bc9a31cc76 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:25:57.064593 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 581/1177] 10.222.0.33 () {60 vars in 1132 bytes} [Mon Jul 25 05:25:56 2022] POST /identity/v3/users => generated 311 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 05:25:59.109836 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da77004f-706c-4455-bc0f-bd5f87a7f506 None None] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:25:59.110410 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da77004f-706c-4455-bc0f-bd5f87a7f506 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:25:59.110410 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da77004f-706c-4455-bc0f-bd5f87a7f506 None None] PATH_INFO: `/` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:25:59.111689 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 597/1178] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jul 25 05:25:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:25:59.116730 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5983a2f-500d-449a-bf00-8874b535ad15 None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:25:59.117166 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5983a2f-500d-449a-bf00-8874b535ad15 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:25:59.117315 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5983a2f-500d-449a-bf00-8874b535ad15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:25:59.133704 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-b5983a2f-500d-449a-bf00-8874b535ad15 None None] MFA Rules not processed for user `1eae540242d94c10b0ad33960a855f08`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:25:59.134876 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-b5983a2f-500d-449a-bf00-8874b535ad15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:25:59.198327 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 582/1179] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jul 25 05:25:59 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 25 05:25:59.221678 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da77004f-706c-4455-bc0f-bd5f87a7f506 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:25:59.295673 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:25:59.322228 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da77004f-706c-4455-bc0f-bd5f87a7f506 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:25:59.354319 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa5eab64-f52f-4a5e-b5bd-c203f9246022 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:25:59.355156 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5eab64-f52f-4a5e-b5bd-c203f9246022 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:25:59.355441 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5eab64-f52f-4a5e-b5bd-c203f9246022 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:25:59.355715 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5eab64-f52f-4a5e-b5bd-c203f9246022 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:25:59.365467 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa5eab64-f52f-4a5e-b5bd-c203f9246022 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 05:25:59.373442 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application [None req-aa5eab64-f52f-4a5e-b5bd-c203f9246022 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 05:25:59.373442 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:25:59.373442 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:25:59.373442 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:25:59.373442 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:25:59.373442 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:25:59.373442 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:25:59.373442 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:25:59.373442 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:25:59.373442 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:25:59.373442 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:25:59.373442 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:25:59.373442 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 05:25:59.373442 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 05:25:59.373442 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 05:25:59.373442 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 05:25:59.374646 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 05:25:59.374646 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 05:25:59.374646 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 05:25:59.374646 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 05:25:59.374646 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:25:59.374646 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:25:59.374646 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:25:59.374646 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:25:59.374646 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:25:59.374646 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:25:59.374646 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:25:59.374646 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application with Lock( Jul 25 05:25:59.374646 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:25:59.374646 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:25:59.374646 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:25:59.374646 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:25:59.375961 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:25:59.375961 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:25:59.375961 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:25:59.375961 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:25:59.375961 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 05:25:59.375961 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 05:25:59.375961 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 05:25:59.375961 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 05:25:59.375961 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 05:25:59.375961 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 05:25:59.375961 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 05:25:59.375961 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Jul 25 05:25:59.375961 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 598/1180] 10.222.0.33 () {56 vars in 1096 bytes} [Mon Jul 25 05:25:59 2022] GET /identity/v3/roles/admin => generated 83 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:25:59.380498 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5983a2f-500d-449a-bf00-8874b535ad15 None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:25:59.458746 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:25:59.478211 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5983a2f-500d-449a-bf00-8874b535ad15 None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:25:59.511258 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81307060-de31-4ebe-9b32-00afd0ace1ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:25:59.511914 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81307060-de31-4ebe-9b32-00afd0ace1ab None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:25:59.512200 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81307060-de31-4ebe-9b32-00afd0ace1ab None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:25:59.512480 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81307060-de31-4ebe-9b32-00afd0ace1ab None admin] PATH_INFO: `/v3/roles` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:25:59.513270 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81307060-de31-4ebe-9b32-00afd0ace1ab None admin] RBAC: Adding query filter params (name=admin) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:25:59.513583 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81307060-de31-4ebe-9b32-00afd0ace1ab None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:25:59.514983 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81307060-de31-4ebe-9b32-00afd0ace1ab None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:25:59.522819 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 583/1181] 10.222.0.33 () {56 vars in 1099 bytes} [Mon Jul 25 05:25:59 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:25:59.529197 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa5eab64-f52f-4a5e-b5bd-c203f9246022 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:25:59.628623 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa5eab64-f52f-4a5e-b5bd-c203f9246022 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:25:59.660229 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6956c470-8008-43d8-b82b-d9705d893b90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:25:59.661112 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6956c470-8008-43d8-b82b-d9705d893b90 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:25:59.661392 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6956c470-8008-43d8-b82b-d9705d893b90 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:25:59.661664 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6956c470-8008-43d8-b82b-d9705d893b90 None admin] PATH_INFO: `/v3/users/60ae41528e3d4ee1a4395c5b11138ee0` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:25:59.663513 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:25:59.675415 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6956c470-8008-43d8-b82b-d9705d893b90 None admin] RBAC: Authorizing `identity:get_user(user_id=60ae41528e3d4ee1a4395c5b11138ee0)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:25:59.676866 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6956c470-8008-43d8-b82b-d9705d893b90 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:25:59.678487 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 599/1182] 10.222.0.33 () {56 vars in 1177 bytes} [Mon Jul 25 05:25:59 2022] GET /identity/v3/users/60ae41528e3d4ee1a4395c5b11138ee0 => generated 311 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:25:59.685239 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81307060-de31-4ebe-9b32-00afd0ace1ab None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:25:59.781394 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81307060-de31-4ebe-9b32-00afd0ace1ab None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:25:59.815601 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea32557-e753-43c8-80f4-9ed571f0c4d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:25:59.816492 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea32557-e753-43c8-80f4-9ed571f0c4d8 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:25:59.816773 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea32557-e753-43c8-80f4-9ed571f0c4d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:25:59.817073 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea32557-e753-43c8-80f4-9ed571f0c4d8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:25:59.830828 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:25:59.881918 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eea32557-e753-43c8-80f4-9ed571f0c4d8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:25:59.883287 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eea32557-e753-43c8-80f4-9ed571f0c4d8 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:25:59.975946 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 584/1183] 10.222.0.33 () {58 vars in 1282 bytes} [Mon Jul 25 05:25:59 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 291 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 25 05:25:59.982493 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6956c470-8008-43d8-b82b-d9705d893b90 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:26:00.078531 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6956c470-8008-43d8-b82b-d9705d893b90 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:26:00.110557 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed6ff0e7-66b0-425f-91f8-751eae6f10d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:26:00.111401 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6ff0e7-66b0-425f-91f8-751eae6f10d2 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:26:00.111684 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6ff0e7-66b0-425f-91f8-751eae6f10d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:26:00.111960 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6ff0e7-66b0-425f-91f8-751eae6f10d2 None admin] PATH_INFO: `/v3/projects/cdb7aceba471473e88d4f65a6e149ea1` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:26:00.113144 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed6ff0e7-66b0-425f-91f8-751eae6f10d2 None admin] RBAC: Authorizing `identity:get_project(project_id=cdb7aceba471473e88d4f65a6e149ea1)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:26:00.114553 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed6ff0e7-66b0-425f-91f8-751eae6f10d2 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:26:00.116225 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 600/1184] 10.222.0.33 () {56 vars in 1186 bytes} [Mon Jul 25 05:25:59 2022] GET /identity/v3/projects/cdb7aceba471473e88d4f65a6e149ea1 => generated 361 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:26:00.123056 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea32557-e753-43c8-80f4-9ed571f0c4d8 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:26:00.166185 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:26:00.220803 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea32557-e753-43c8-80f4-9ed571f0c4d8 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:26:00.254217 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c2d9af6-bc9b-43d6-877c-ac36b97f7330 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:26:00.254866 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2d9af6-bc9b-43d6-877c-ac36b97f7330 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:26:00.255152 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2d9af6-bc9b-43d6-877c-ac36b97f7330 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:26:00.255431 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2d9af6-bc9b-43d6-877c-ac36b97f7330 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:26:00.256261 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c2d9af6-bc9b-43d6-877c-ac36b97f7330 None admin] RBAC: Adding query filter params (role.id=39e5c666285a42fe8fc2c3afca6260d1, scope.project.id=cdb7aceba471473e88d4f65a6e149ea1, user.id=60ae41528e3d4ee1a4395c5b11138ee0) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:26:00.256577 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c2d9af6-bc9b-43d6-877c-ac36b97f7330 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:26:00.258103 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c2d9af6-bc9b-43d6-877c-ac36b97f7330 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:26:00.263789 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 585/1185] 10.222.0.33 () {56 vars in 1374 bytes} [Mon Jul 25 05:26:00 2022] GET /identity/v3/role_assignments?user.id=60ae41528e3d4ee1a4395c5b11138ee0&scope.project.id=cdb7aceba471473e88d4f65a6e149ea1&role.id=39e5c666285a42fe8fc2c3afca6260d1 => generated 259 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:26:02.252713 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f09769-200f-40b8-b4dc-dc1a35294b17 None None] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:26:02.253271 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f09769-200f-40b8-b4dc-dc1a35294b17 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:26:02.253271 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f09769-200f-40b8-b4dc-dc1a35294b17 None None] PATH_INFO: `/` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:26:02.254609 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 601/1186] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jul 25 05:26:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:26:02.259771 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ad74fc-39b1-432b-92f8-a2f3fa6f4c29 None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:26:02.260811 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ad74fc-39b1-432b-92f8-a2f3fa6f4c29 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:26:02.260898 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ad74fc-39b1-432b-92f8-a2f3fa6f4c29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:26:02.278336 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-01ad74fc-39b1-432b-92f8-a2f3fa6f4c29 None None] MFA Rules not processed for user `1eae540242d94c10b0ad33960a855f08`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:26:02.279574 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-01ad74fc-39b1-432b-92f8-a2f3fa6f4c29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:26:02.348584 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 586/1187] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jul 25 05:26:02 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 25 05:26:02.371370 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f09769-200f-40b8-b4dc-dc1a35294b17 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:26:02.430948 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:26:02.472450 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f09769-200f-40b8-b4dc-dc1a35294b17 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:26:02.505494 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7571f809-fdaa-4850-a2cc-44afbd1e30dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:26:02.506259 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7571f809-fdaa-4850-a2cc-44afbd1e30dc None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:26:02.506496 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7571f809-fdaa-4850-a2cc-44afbd1e30dc None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:26:02.506776 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7571f809-fdaa-4850-a2cc-44afbd1e30dc None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:26:02.517472 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7571f809-fdaa-4850-a2cc-44afbd1e30dc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 05:26:02.525055 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application [None req-7571f809-fdaa-4850-a2cc-44afbd1e30dc None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 05:26:02.525055 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:26:02.525055 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:26:02.525055 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:26:02.525055 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:26:02.525055 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:26:02.525055 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:26:02.525055 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:26:02.525055 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:26:02.525055 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:26:02.525055 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:26:02.525055 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:26:02.525055 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 05:26:02.525055 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 05:26:02.525055 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 05:26:02.525055 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 05:26:02.526396 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 05:26:02.526396 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 05:26:02.526396 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 05:26:02.526396 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 05:26:02.526396 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:26:02.526396 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:26:02.526396 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:26:02.526396 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:26:02.526396 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:26:02.526396 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:26:02.526396 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:26:02.526396 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application with Lock( Jul 25 05:26:02.526396 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:26:02.526396 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:26:02.526396 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:26:02.526396 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:26:02.527772 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:26:02.527772 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:26:02.527772 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:26:02.527772 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:26:02.527772 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 05:26:02.527772 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 05:26:02.527772 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 05:26:02.527772 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 05:26:02.527772 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 05:26:02.527772 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 05:26:02.527772 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 05:26:02.527772 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Jul 25 05:26:02.527772 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 602/1188] 10.222.0.33 () {56 vars in 1096 bytes} [Mon Jul 25 05:26:02 2022] GET /identity/v3/roles/admin => generated 83 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:26:02.532648 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ad74fc-39b1-432b-92f8-a2f3fa6f4c29 None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:26:02.570701 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:26:02.628178 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ad74fc-39b1-432b-92f8-a2f3fa6f4c29 None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:26:02.660716 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e62e25b-a2c4-4970-af16-56cedf1e7cdf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:26:02.661406 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e62e25b-a2c4-4970-af16-56cedf1e7cdf None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:26:02.661691 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e62e25b-a2c4-4970-af16-56cedf1e7cdf None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:26:02.661968 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e62e25b-a2c4-4970-af16-56cedf1e7cdf None admin] PATH_INFO: `/v3/roles` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:26:02.662716 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e62e25b-a2c4-4970-af16-56cedf1e7cdf None admin] RBAC: Adding query filter params (name=admin) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:26:02.663022 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e62e25b-a2c4-4970-af16-56cedf1e7cdf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:26:02.664382 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e62e25b-a2c4-4970-af16-56cedf1e7cdf None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:26:02.672230 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 587/1189] 10.222.0.33 () {56 vars in 1099 bytes} [Mon Jul 25 05:26:02 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:26:02.678246 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7571f809-fdaa-4850-a2cc-44afbd1e30dc None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:26:02.774728 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7571f809-fdaa-4850-a2cc-44afbd1e30dc None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:26:02.793963 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:26:02.808528 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b0abde7-ca7a-4622-8004-44e88a2c4b4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:26:02.809427 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0abde7-ca7a-4622-8004-44e88a2c4b4c None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:26:02.809713 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0abde7-ca7a-4622-8004-44e88a2c4b4c None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:26:02.809988 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0abde7-ca7a-4622-8004-44e88a2c4b4c None admin] PATH_INFO: `/v3/users/60ae41528e3d4ee1a4395c5b11138ee0` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:26:02.811183 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b0abde7-ca7a-4622-8004-44e88a2c4b4c None admin] RBAC: Authorizing `identity:get_user(user_id=60ae41528e3d4ee1a4395c5b11138ee0)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:26:02.812606 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b0abde7-ca7a-4622-8004-44e88a2c4b4c None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:26:02.814271 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 603/1190] 10.222.0.33 () {56 vars in 1177 bytes} [Mon Jul 25 05:26:02 2022] GET /identity/v3/users/60ae41528e3d4ee1a4395c5b11138ee0 => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:26:02.820437 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e62e25b-a2c4-4970-af16-56cedf1e7cdf None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:26:02.917267 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:26:02.920395 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e62e25b-a2c4-4970-af16-56cedf1e7cdf None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:26:02.953501 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb5b2078-4697-4a02-b367-f91fc82fc37a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:26:02.954353 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5b2078-4697-4a02-b367-f91fc82fc37a None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:26:02.954641 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5b2078-4697-4a02-b367-f91fc82fc37a None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:26:02.954921 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5b2078-4697-4a02-b367-f91fc82fc37a None admin] PATH_INFO: `/v3/projects/cdb7aceba471473e88d4f65a6e149ea1` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:26:02.956092 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb5b2078-4697-4a02-b367-f91fc82fc37a None admin] RBAC: Authorizing `identity:get_project(project_id=cdb7aceba471473e88d4f65a6e149ea1)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:26:02.957648 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb5b2078-4697-4a02-b367-f91fc82fc37a None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:26:02.959345 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 588/1191] 10.222.0.33 () {56 vars in 1186 bytes} [Mon Jul 25 05:26:02 2022] GET /identity/v3/projects/cdb7aceba471473e88d4f65a6e149ea1 => generated 361 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:26:02.965249 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b0abde7-ca7a-4622-8004-44e88a2c4b4c None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:26:03.061997 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b0abde7-ca7a-4622-8004-44e88a2c4b4c None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:26:03.094057 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-952567d3-9489-4c4d-bd0e-250df7794f92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:26:03.094931 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952567d3-9489-4c4d-bd0e-250df7794f92 None admin] REQUEST_METHOD: `PUT` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:26:03.095218 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952567d3-9489-4c4d-bd0e-250df7794f92 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:26:03.095499 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952567d3-9489-4c4d-bd0e-250df7794f92 None admin] PATH_INFO: `/v3/projects/cdb7aceba471473e88d4f65a6e149ea1/users/60ae41528e3d4ee1a4395c5b11138ee0/roles/39e5c666285a42fe8fc2c3afca6260d1` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:26:03.097467 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-952567d3-9489-4c4d-bd0e-250df7794f92 None admin] RBAC: Authorizing `identity:create_grant(project_id=cdb7aceba471473e88d4f65a6e149ea1, user_id=60ae41528e3d4ee1a4395c5b11138ee0, role_id=39e5c666285a42fe8fc2c3afca6260d1)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:26:03.098885 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-952567d3-9489-4c4d-bd0e-250df7794f92 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:26:03.112446 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 604/1192] 10.222.0.33 () {58 vars in 1439 bytes} [Mon Jul 25 05:26:02 2022] PUT /identity/v3/projects/cdb7aceba471473e88d4f65a6e149ea1/users/60ae41528e3d4ee1a4395c5b11138ee0/roles/39e5c666285a42fe8fc2c3afca6260d1 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 05:26:05.165522 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644ee977-a81b-432f-bd3d-0cc5a3d40bb5 None None] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:26:05.166054 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644ee977-a81b-432f-bd3d-0cc5a3d40bb5 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:26:05.166054 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644ee977-a81b-432f-bd3d-0cc5a3d40bb5 None None] PATH_INFO: `/` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:26:05.167976 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 589/1193] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jul 25 05:26:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:26:05.173110 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1238558a-c57f-4600-99e3-866eae208580 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:26:05.173540 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1238558a-c57f-4600-99e3-866eae208580 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:26:05.173666 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1238558a-c57f-4600-99e3-866eae208580 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:26:05.190742 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-1238558a-c57f-4600-99e3-866eae208580 None None] MFA Rules not processed for user `1eae540242d94c10b0ad33960a855f08`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:26:05.192168 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-1238558a-c57f-4600-99e3-866eae208580 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:26:05.234740 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:26:05.259725 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 605/1194] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jul 25 05:26:05 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 25 05:26:05.283321 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-644ee977-a81b-432f-bd3d-0cc5a3d40bb5 None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:26:05.381963 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-644ee977-a81b-432f-bd3d-0cc5a3d40bb5 None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:26:05.413853 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef903a8d-1a04-4f44-a07e-4dd359375879 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:26:05.414628 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef903a8d-1a04-4f44-a07e-4dd359375879 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:26:05.414910 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef903a8d-1a04-4f44-a07e-4dd359375879 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:26:05.415183 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef903a8d-1a04-4f44-a07e-4dd359375879 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:26:05.425453 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef903a8d-1a04-4f44-a07e-4dd359375879 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 05:26:05.433527 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application [None req-ef903a8d-1a04-4f44-a07e-4dd359375879 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 05:26:05.433527 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:26:05.433527 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:26:05.433527 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:26:05.433527 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:26:05.433527 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:26:05.433527 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:26:05.433527 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:26:05.433527 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:26:05.433527 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:26:05.433527 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:26:05.433527 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:26:05.433527 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 05:26:05.433527 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 05:26:05.433527 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 05:26:05.433527 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 05:26:05.434642 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 05:26:05.434642 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 05:26:05.434642 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 05:26:05.434642 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 05:26:05.434642 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:26:05.434642 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:26:05.434642 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:26:05.434642 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:26:05.434642 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:26:05.434642 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:26:05.434642 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:26:05.434642 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application with Lock( Jul 25 05:26:05.434642 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:26:05.434642 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:26:05.434642 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:26:05.434642 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:26:05.435690 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:26:05.435690 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:26:05.435690 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:26:05.435690 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:26:05.435690 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 05:26:05.435690 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 05:26:05.435690 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 05:26:05.435690 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 05:26:05.435690 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 05:26:05.435690 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 05:26:05.435690 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 05:26:05.435690 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Jul 25 05:26:05.435690 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 590/1195] 10.222.0.33 () {56 vars in 1096 bytes} [Mon Jul 25 05:26:05 2022] GET /identity/v3/roles/admin => generated 83 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:26:05.440582 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1238558a-c57f-4600-99e3-866eae208580 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:26:05.537900 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1238558a-c57f-4600-99e3-866eae208580 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:26:05.571400 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b49e2da-5c6f-4e80-9a3f-52213b277741 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:26:05.572056 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b49e2da-5c6f-4e80-9a3f-52213b277741 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:26:05.572344 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b49e2da-5c6f-4e80-9a3f-52213b277741 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:26:05.573228 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b49e2da-5c6f-4e80-9a3f-52213b277741 None admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:26:05.574270 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b49e2da-5c6f-4e80-9a3f-52213b277741 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:26:05.574582 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b49e2da-5c6f-4e80-9a3f-52213b277741 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:26:05.575977 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b49e2da-5c6f-4e80-9a3f-52213b277741 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:26:05.583808 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 606/1196] 10.222.0.33 () {56 vars in 1099 bytes} [Mon Jul 25 05:26:05 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:26:05.589768 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef903a8d-1a04-4f44-a07e-4dd359375879 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:26:05.617213 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:26:05.686239 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef903a8d-1a04-4f44-a07e-4dd359375879 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:26:05.718891 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4414d79f-e620-4221-bebc-cb6111d6d3c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:26:05.719746 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4414d79f-e620-4221-bebc-cb6111d6d3c9 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:26:05.720031 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4414d79f-e620-4221-bebc-cb6111d6d3c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:26:05.720307 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4414d79f-e620-4221-bebc-cb6111d6d3c9 None admin] PATH_INFO: `/v3/users/60ae41528e3d4ee1a4395c5b11138ee0` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:26:05.721498 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4414d79f-e620-4221-bebc-cb6111d6d3c9 None admin] RBAC: Authorizing `identity:get_user(user_id=60ae41528e3d4ee1a4395c5b11138ee0)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:26:05.722924 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4414d79f-e620-4221-bebc-cb6111d6d3c9 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:26:05.724526 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 591/1197] 10.222.0.33 () {56 vars in 1177 bytes} [Mon Jul 25 05:26:05 2022] GET /identity/v3/users/60ae41528e3d4ee1a4395c5b11138ee0 => generated 311 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:26:05.730292 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b49e2da-5c6f-4e80-9a3f-52213b277741 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:26:05.771844 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:26:05.831136 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b49e2da-5c6f-4e80-9a3f-52213b277741 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:26:05.865592 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f4c3494-1c61-4194-89c6-f07118da33a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:26:05.866239 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4c3494-1c61-4194-89c6-f07118da33a9 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:26:05.866529 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4c3494-1c61-4194-89c6-f07118da33a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:26:05.866811 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4c3494-1c61-4194-89c6-f07118da33a9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:26:05.931292 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f4c3494-1c61-4194-89c6-f07118da33a9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:26:05.933363 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f4c3494-1c61-4194-89c6-f07118da33a9 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:26:05.971487 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:26:06.028093 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 607/1198] 10.222.0.33 () {58 vars in 1282 bytes} [Mon Jul 25 05:26:05 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 298 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 25 05:26:06.034006 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4414d79f-e620-4221-bebc-cb6111d6d3c9 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:26:06.124342 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:26:06.130250 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4414d79f-e620-4221-bebc-cb6111d6d3c9 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:26:06.163890 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87492030-9336-4aaf-b682-0e1023f1c58a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:26:06.164729 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87492030-9336-4aaf-b682-0e1023f1c58a None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:26:06.165045 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87492030-9336-4aaf-b682-0e1023f1c58a None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:26:06.165321 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87492030-9336-4aaf-b682-0e1023f1c58a None admin] PATH_INFO: `/v3/projects/cdb7aceba471473e88d4f65a6e149ea1` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:26:06.167097 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87492030-9336-4aaf-b682-0e1023f1c58a None admin] RBAC: Authorizing `identity:get_project(project_id=cdb7aceba471473e88d4f65a6e149ea1)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:26:06.168508 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87492030-9336-4aaf-b682-0e1023f1c58a None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:26:06.170219 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 592/1199] 10.222.0.33 () {56 vars in 1186 bytes} [Mon Jul 25 05:26:06 2022] GET /identity/v3/projects/cdb7aceba471473e88d4f65a6e149ea1 => generated 361 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:26:06.176219 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f4c3494-1c61-4194-89c6-f07118da33a9 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:26:06.271745 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f4c3494-1c61-4194-89c6-f07118da33a9 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:26:06.304129 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5293cc6-e78c-4e72-8a1b-adad39a64720 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:26:06.304765 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5293cc6-e78c-4e72-8a1b-adad39a64720 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:26:06.305077 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5293cc6-e78c-4e72-8a1b-adad39a64720 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:26:06.305349 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5293cc6-e78c-4e72-8a1b-adad39a64720 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:26:06.306172 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5293cc6-e78c-4e72-8a1b-adad39a64720 None admin] RBAC: Adding query filter params (role.id=39e5c666285a42fe8fc2c3afca6260d1, scope.project.id=cdb7aceba471473e88d4f65a6e149ea1, user.id=60ae41528e3d4ee1a4395c5b11138ee0) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:26:06.306480 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5293cc6-e78c-4e72-8a1b-adad39a64720 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:26:06.308532 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5293cc6-e78c-4e72-8a1b-adad39a64720 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:26:06.314175 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 608/1200] 10.222.0.33 () {56 vars in 1374 bytes} [Mon Jul 25 05:26:06 2022] GET /identity/v3/role_assignments?user.id=60ae41528e3d4ee1a4395c5b11138ee0&scope.project.id=cdb7aceba471473e88d4f65a6e149ea1&role.id=39e5c666285a42fe8fc2c3afca6260d1 => generated 608 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:26:16.160379 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=18 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:26:16.311610 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=98 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:27:01.668893 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf954951-643d-4285-85e4-e19f666089fc None None] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:27:01.669498 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf954951-643d-4285-85e4-e19f666089fc None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:27:01.669498 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf954951-643d-4285-85e4-e19f666089fc None None] PATH_INFO: `/` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:27:01.670832 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 593/1201] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jul 25 05:27:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:27:01.676270 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb1f863-3450-4db4-8645-e4041417cb08 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:27:01.676738 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb1f863-3450-4db4-8645-e4041417cb08 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:27:01.676870 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb1f863-3450-4db4-8645-e4041417cb08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:27:01.695512 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-aeb1f863-3450-4db4-8645-e4041417cb08 None None] MFA Rules not processed for user `1eae540242d94c10b0ad33960a855f08`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:27:01.696737 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-aeb1f863-3450-4db4-8645-e4041417cb08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:27:01.777225 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 609/1202] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jul 25 05:27:01 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 25 05:27:01.801988 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf954951-643d-4285-85e4-e19f666089fc None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:27:01.950523 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf954951-643d-4285-85e4-e19f666089fc None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:27:01.994283 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3394acc-97de-4cf6-a37f-91f6d846ac51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:27:01.995307 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3394acc-97de-4cf6-a37f-91f6d846ac51 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:27:01.995626 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3394acc-97de-4cf6-a37f-91f6d846ac51 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:27:01.995896 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3394acc-97de-4cf6-a37f-91f6d846ac51 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:27:02.011417 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3394acc-97de-4cf6-a37f-91f6d846ac51 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:27:02.013264 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3394acc-97de-4cf6-a37f-91f6d846ac51 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:27:02.032581 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application [None req-a3394acc-97de-4cf6-a37f-91f6d846ac51 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 05:27:02.032581 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:27:02.032581 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:27:02.032581 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:27:02.032581 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:27:02.032581 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:27:02.032581 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:27:02.032581 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:27:02.032581 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:27:02.032581 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:27:02.032581 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:27:02.032581 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:27:02.032581 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 05:27:02.032581 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 05:27:02.032581 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 05:27:02.033673 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 05:27:02.033673 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:27:02.033673 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:27:02.033673 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:27:02.033673 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:27:02.033673 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:27:02.033673 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:27:02.033673 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:27:02.033673 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application with Lock( Jul 25 05:27:02.033673 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:27:02.033673 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:27:02.033673 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:27:02.033673 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:27:02.033673 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:27:02.033673 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:27:02.033673 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:27:02.033673 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:27:02.034751 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 05:27:02.034751 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 05:27:02.034751 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 05:27:02.034751 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Jul 25 05:27:02.034751 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 594/1203] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Jul 25 05:27:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:27:02.042464 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb1f863-3450-4db4-8645-e4041417cb08 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:27:02.173305 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb1f863-3450-4db4-8645-e4041417cb08 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:27:02.207767 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:27:02.215162 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47fad99a-1317-42e4-83f7-b05a1d84b9d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:27:02.215837 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47fad99a-1317-42e4-83f7-b05a1d84b9d7 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:27:02.216188 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47fad99a-1317-42e4-83f7-b05a1d84b9d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:27:02.216443 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47fad99a-1317-42e4-83f7-b05a1d84b9d7 None admin] PATH_INFO: `/v3/domains` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:27:02.217296 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47fad99a-1317-42e4-83f7-b05a1d84b9d7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:27:02.217591 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47fad99a-1317-42e4-83f7-b05a1d84b9d7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:27:02.219677 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47fad99a-1317-42e4-83f7-b05a1d84b9d7 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:27:02.233080 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 610/1204] 10.222.0.33 () {56 vars in 1109 bytes} [Mon Jul 25 05:27:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 05:27:02.241575 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3394acc-97de-4cf6-a37f-91f6d846ac51 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:27:02.311865 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:27:02.372712 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3394acc-97de-4cf6-a37f-91f6d846ac51 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:27:02.420250 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e140a22d-346e-4315-89c9-75cb97f5128f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:27:02.420988 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e140a22d-346e-4315-89c9-75cb97f5128f None admin] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:27:02.421278 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e140a22d-346e-4315-89c9-75cb97f5128f None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:27:02.421569 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e140a22d-346e-4315-89c9-75cb97f5128f None admin] PATH_INFO: `/v3/users` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:27:02.422393 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e140a22d-346e-4315-89c9-75cb97f5128f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:27:02.424059 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e140a22d-346e-4315-89c9-75cb97f5128f None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:27:02.451299 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 595/1205] 10.222.0.33 () {60 vars in 1132 bytes} [Mon Jul 25 05:27:02 2022] POST /identity/v3/users => generated 251 bytes in 210 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 05:27:05.031529 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d2689d-b8e5-4e32-96d6-e807e4df324b None None] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:27:05.032064 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d2689d-b8e5-4e32-96d6-e807e4df324b None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:27:05.032064 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d2689d-b8e5-4e32-96d6-e807e4df324b None None] PATH_INFO: `/` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:27:05.033661 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 611/1206] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jul 25 05:27:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:27:05.041303 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70546b8c-c67e-4f07-9cdf-880312a2202e None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:27:05.041756 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70546b8c-c67e-4f07-9cdf-880312a2202e None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:27:05.041850 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70546b8c-c67e-4f07-9cdf-880312a2202e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:27:05.061148 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-70546b8c-c67e-4f07-9cdf-880312a2202e None None] MFA Rules not processed for user `1eae540242d94c10b0ad33960a855f08`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:27:05.062286 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-70546b8c-c67e-4f07-9cdf-880312a2202e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:27:05.146071 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 596/1207] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jul 25 05:27:05 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 25 05:27:05.170085 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d2689d-b8e5-4e32-96d6-e807e4df324b None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:27:05.284713 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d2689d-b8e5-4e32-96d6-e807e4df324b None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:27:05.325856 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32698dd3-ff40-4806-8b4a-07ff042d38e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:27:05.326583 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32698dd3-ff40-4806-8b4a-07ff042d38e4 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:27:05.326966 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32698dd3-ff40-4806-8b4a-07ff042d38e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:27:05.327151 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32698dd3-ff40-4806-8b4a-07ff042d38e4 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:27:05.341152 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32698dd3-ff40-4806-8b4a-07ff042d38e4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 05:27:05.349844 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application [None req-32698dd3-ff40-4806-8b4a-07ff042d38e4 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 25 05:27:05.349844 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:27:05.349844 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:27:05.349844 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:27:05.349844 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:27:05.349844 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:27:05.349844 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:27:05.349844 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:27:05.349844 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:27:05.349844 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:27:05.349844 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:27:05.349844 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:27:05.349844 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 05:27:05.349844 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 05:27:05.349844 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 05:27:05.349844 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 05:27:05.350917 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 05:27:05.350917 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 05:27:05.350917 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 05:27:05.350917 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 05:27:05.350917 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:27:05.350917 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:27:05.350917 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:27:05.350917 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:27:05.350917 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:27:05.350917 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:27:05.350917 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:27:05.350917 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application with Lock( Jul 25 05:27:05.350917 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:27:05.350917 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:27:05.350917 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:27:05.350917 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:27:05.351805 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:27:05.351805 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:27:05.351805 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:27:05.351805 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:27:05.351805 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 05:27:05.351805 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 05:27:05.351805 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 05:27:05.351805 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 05:27:05.351805 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 05:27:05.351805 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 05:27:05.351805 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 25 05:27:05.351805 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Jul 25 05:27:05.351805 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 612/1208] 10.222.0.33 () {56 vars in 1102 bytes} [Mon Jul 25 05:27:05 2022] GET /identity/v3/roles/service => generated 85 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:27:05.357210 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70546b8c-c67e-4f07-9cdf-880312a2202e None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:27:05.399649 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:27:05.471957 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70546b8c-c67e-4f07-9cdf-880312a2202e None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:27:05.515678 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e0fbfe1-5d00-4200-8fc0-5fb9629f8f6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:27:05.516360 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0fbfe1-5d00-4200-8fc0-5fb9629f8f6d None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:27:05.516655 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0fbfe1-5d00-4200-8fc0-5fb9629f8f6d None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:27:05.516965 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0fbfe1-5d00-4200-8fc0-5fb9629f8f6d None admin] PATH_INFO: `/v3/roles` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:27:05.517727 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e0fbfe1-5d00-4200-8fc0-5fb9629f8f6d None admin] RBAC: Adding query filter params (name=service) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:27:05.518039 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e0fbfe1-5d00-4200-8fc0-5fb9629f8f6d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:27:05.519414 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e0fbfe1-5d00-4200-8fc0-5fb9629f8f6d None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:27:05.532338 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 597/1209] 10.222.0.33 () {56 vars in 1103 bytes} [Mon Jul 25 05:27:05 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 05:27:05.534094 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32698dd3-ff40-4806-8b4a-07ff042d38e4 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:27:05.591378 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:27:05.649517 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32698dd3-ff40-4806-8b4a-07ff042d38e4 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:27:05.685171 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93659355-de03-43e5-ad6e-025bbb7db93a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:27:05.686052 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93659355-de03-43e5-ad6e-025bbb7db93a None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:27:05.686389 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93659355-de03-43e5-ad6e-025bbb7db93a None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:27:05.686644 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93659355-de03-43e5-ad6e-025bbb7db93a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:27:05.698139 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93659355-de03-43e5-ad6e-025bbb7db93a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:27:05.700475 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93659355-de03-43e5-ad6e-025bbb7db93a None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:27:05.718083 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application [None req-93659355-de03-43e5-ad6e-025bbb7db93a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 05:27:05.718083 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:27:05.718083 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:27:05.718083 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:27:05.718083 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:27:05.718083 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:27:05.718083 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:27:05.718083 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:27:05.718083 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:27:05.718083 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:27:05.718083 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:27:05.718083 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:27:05.718083 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 05:27:05.718083 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 05:27:05.718083 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 05:27:05.719257 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 05:27:05.719257 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:27:05.719257 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:27:05.719257 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:27:05.719257 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:27:05.719257 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:27:05.719257 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:27:05.719257 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:27:05.719257 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application with Lock( Jul 25 05:27:05.719257 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:27:05.719257 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:27:05.719257 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:27:05.719257 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:27:05.719257 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:27:05.719257 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:27:05.719257 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:27:05.719257 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:27:05.720455 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 05:27:05.720455 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 05:27:05.720455 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 05:27:05.720455 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Jul 25 05:27:05.725428 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 613/1210] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Jul 25 05:27:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:27:05.725583 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e0fbfe1-5d00-4200-8fc0-5fb9629f8f6d None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:27:05.848104 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:27:05.851549 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e0fbfe1-5d00-4200-8fc0-5fb9629f8f6d None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:27:05.892811 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f5f2ed-f0af-4e23-a14f-621a489f16ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:27:05.892811 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f5f2ed-f0af-4e23-a14f-621a489f16ad None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:27:05.892811 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f5f2ed-f0af-4e23-a14f-621a489f16ad None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:27:05.892811 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f5f2ed-f0af-4e23-a14f-621a489f16ad None admin] PATH_INFO: `/v3/domains` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:27:05.893262 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60f5f2ed-f0af-4e23-a14f-621a489f16ad None admin] RBAC: Adding query filter params (name=Default) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:27:05.893262 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60f5f2ed-f0af-4e23-a14f-621a489f16ad None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:27:05.893640 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60f5f2ed-f0af-4e23-a14f-621a489f16ad None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:27:05.908833 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 598/1211] 10.222.0.33 () {56 vars in 1109 bytes} [Mon Jul 25 05:27:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:27:05.914480 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93659355-de03-43e5-ad6e-025bbb7db93a None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:27:06.039796 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93659355-de03-43e5-ad6e-025bbb7db93a None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:27:06.041513 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:27:06.079198 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b0313dc-cc94-47a7-990a-093e891e0e90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:27:06.080012 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0313dc-cc94-47a7-990a-093e891e0e90 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:27:06.080307 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0313dc-cc94-47a7-990a-093e891e0e90 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:27:06.080587 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0313dc-cc94-47a7-990a-093e891e0e90 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:27:06.091155 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b0313dc-cc94-47a7-990a-093e891e0e90 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:27:06.092638 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b0313dc-cc94-47a7-990a-093e891e0e90 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:27:06.109522 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application [None req-8b0313dc-cc94-47a7-990a-093e891e0e90 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 25 05:27:06.109522 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:27:06.109522 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:27:06.109522 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:27:06.109522 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:27:06.109522 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:27:06.109522 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:27:06.109522 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:27:06.109522 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:27:06.109522 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:27:06.109522 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:27:06.109522 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:27:06.109522 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 25 05:27:06.109522 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 25 05:27:06.109522 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 25 05:27:06.109522 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 25 05:27:06.110728 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:27:06.110728 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:27:06.110728 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 25 05:27:06.110728 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 05:27:06.110728 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 25 05:27:06.110728 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 05:27:06.110728 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:27:06.110728 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:27:06.110728 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:27:06.110728 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:27:06.110728 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:27:06.110728 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application with Lock( Jul 25 05:27:06.110728 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:27:06.110728 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:27:06.110728 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:27:06.110728 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:27:06.110728 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:27:06.110728 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:27:06.111905 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:27:06.111905 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:27:06.111905 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 25 05:27:06.111905 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 25 05:27:06.111905 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 25 05:27:06.111905 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 25 05:27:06.111905 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 25 05:27:06.111905 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 25 05:27:06.111905 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 25 05:27:06.111905 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Jul 25 05:27:06.112842 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 614/1212] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Jul 25 05:27:05 2022] GET /identity/v3/users/placement => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:27:06.118549 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f5f2ed-f0af-4e23-a14f-621a489f16ad None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:27:06.234609 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f5f2ed-f0af-4e23-a14f-621a489f16ad None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:27:06.275054 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b2d7ca-9bba-4311-af38-7718d5c80889 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:27:06.276006 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b2d7ca-9bba-4311-af38-7718d5c80889 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:27:06.276340 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b2d7ca-9bba-4311-af38-7718d5c80889 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:27:06.276814 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b2d7ca-9bba-4311-af38-7718d5c80889 None admin] PATH_INFO: `/v3/users` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:27:06.277996 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25b2d7ca-9bba-4311-af38-7718d5c80889 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:27:06.278452 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25b2d7ca-9bba-4311-af38-7718d5c80889 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:27:06.280732 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25b2d7ca-9bba-4311-af38-7718d5c80889 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:27:06.306892 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 599/1213] 10.222.0.33 () {56 vars in 1143 bytes} [Mon Jul 25 05:27:06 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:27:06.309212 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b0313dc-cc94-47a7-990a-093e891e0e90 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:27:06.433837 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b0313dc-cc94-47a7-990a-093e891e0e90 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:27:06.476187 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a39eecf0-c66e-42ca-84de-0f2fc0dadfe5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:27:06.477028 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39eecf0-c66e-42ca-84de-0f2fc0dadfe5 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:27:06.477321 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39eecf0-c66e-42ca-84de-0f2fc0dadfe5 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:27:06.477606 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39eecf0-c66e-42ca-84de-0f2fc0dadfe5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:27:06.508711 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:27:06.557779 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a39eecf0-c66e-42ca-84de-0f2fc0dadfe5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:27:06.559984 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a39eecf0-c66e-42ca-84de-0f2fc0dadfe5 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:27:06.689820 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 615/1214] 10.222.0.33 () {58 vars in 1282 bytes} [Mon Jul 25 05:27:06 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 382 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 25 05:27:06.695255 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b2d7ca-9bba-4311-af38-7718d5c80889 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:27:06.720920 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:27:06.839597 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b2d7ca-9bba-4311-af38-7718d5c80889 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:27:06.883025 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42bf79a8-e8ac-4912-8e7e-17787338dfaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:27:06.884043 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42bf79a8-e8ac-4912-8e7e-17787338dfaf None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:27:06.884313 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42bf79a8-e8ac-4912-8e7e-17787338dfaf None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:27:06.884588 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42bf79a8-e8ac-4912-8e7e-17787338dfaf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:27:06.904013 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42bf79a8-e8ac-4912-8e7e-17787338dfaf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:27:06.905528 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42bf79a8-e8ac-4912-8e7e-17787338dfaf None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:27:06.927380 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application [None req-42bf79a8-e8ac-4912-8e7e-17787338dfaf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 05:27:06.927380 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:27:06.927380 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:27:06.927380 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:27:06.927380 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:27:06.927380 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:27:06.927380 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:27:06.927380 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:27:06.927380 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:27:06.927380 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:27:06.927380 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:27:06.927380 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:27:06.927380 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 05:27:06.927380 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 05:27:06.927380 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 05:27:06.930337 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 05:27:06.930337 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:27:06.930337 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:27:06.930337 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:27:06.930337 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:27:06.930337 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:27:06.930337 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:27:06.930337 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:27:06.930337 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application with Lock( Jul 25 05:27:06.930337 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:27:06.930337 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:27:06.930337 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:27:06.930337 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:27:06.930337 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:27:06.930337 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:27:06.930337 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:27:06.930337 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:27:06.931462 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 05:27:06.931462 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 05:27:06.931462 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 05:27:06.931462 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Jul 25 05:27:06.931462 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 600/1215] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Jul 25 05:27:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:27:06.934693 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a39eecf0-c66e-42ca-84de-0f2fc0dadfe5 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:27:07.006039 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:27:07.049601 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a39eecf0-c66e-42ca-84de-0f2fc0dadfe5 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:27:07.085812 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce382d7f-191a-4cc5-a490-a58e069c08e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:27:07.086507 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce382d7f-191a-4cc5-a490-a58e069c08e9 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:27:07.086807 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce382d7f-191a-4cc5-a490-a58e069c08e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:27:07.087082 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce382d7f-191a-4cc5-a490-a58e069c08e9 None admin] PATH_INFO: `/v3/domains` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:27:07.087843 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce382d7f-191a-4cc5-a490-a58e069c08e9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:27:07.088156 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce382d7f-191a-4cc5-a490-a58e069c08e9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:27:07.090289 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce382d7f-191a-4cc5-a490-a58e069c08e9 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:27:07.108490 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 616/1216] 10.222.0.33 () {56 vars in 1109 bytes} [Mon Jul 25 05:27:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:27:07.109772 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42bf79a8-e8ac-4912-8e7e-17787338dfaf None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:27:07.195104 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:27:07.232377 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42bf79a8-e8ac-4912-8e7e-17787338dfaf None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:27:07.282044 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da7bc4d-7c6e-47d1-9cdc-3ee887bfdca5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:27:07.282759 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da7bc4d-7c6e-47d1-9cdc-3ee887bfdca5 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:27:07.283045 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da7bc4d-7c6e-47d1-9cdc-3ee887bfdca5 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:27:07.283322 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da7bc4d-7c6e-47d1-9cdc-3ee887bfdca5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:27:07.290837 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7da7bc4d-7c6e-47d1-9cdc-3ee887bfdca5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:27:07.293121 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7da7bc4d-7c6e-47d1-9cdc-3ee887bfdca5 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:27:07.306901 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application [None req-7da7bc4d-7c6e-47d1-9cdc-3ee887bfdca5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 05:27:07.306901 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:27:07.306901 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:27:07.306901 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:27:07.306901 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:27:07.306901 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:27:07.306901 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:27:07.306901 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:27:07.306901 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:27:07.306901 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:27:07.306901 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:27:07.306901 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:27:07.306901 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 25 05:27:07.306901 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 25 05:27:07.306901 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 25 05:27:07.308523 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 25 05:27:07.308523 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:27:07.308523 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:27:07.308523 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:27:07.308523 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:27:07.308523 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:27:07.308523 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:27:07.308523 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:27:07.308523 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application with Lock( Jul 25 05:27:07.308523 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:27:07.308523 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:27:07.308523 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:27:07.308523 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:27:07.308523 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:27:07.308523 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:27:07.308523 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:27:07.308523 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:27:07.314422 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 25 05:27:07.314422 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 25 05:27:07.314422 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 25 05:27:07.314422 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 25 05:27:07.314422 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 25 05:27:07.314422 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 25 05:27:07.314422 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 05:27:07.314422 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Jul 25 05:27:07.314422 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 601/1217] 10.222.0.33 () {56 vars in 1111 bytes} [Mon Jul 25 05:27:07 2022] GET /identity/v3/projects/service => generated 88 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:27:07.318701 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce382d7f-191a-4cc5-a490-a58e069c08e9 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:27:07.432029 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce382d7f-191a-4cc5-a490-a58e069c08e9 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:27:07.457135 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:27:07.473788 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-760e770c-0ef9-4321-90a4-c8c21af06a15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:27:07.474410 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760e770c-0ef9-4321-90a4-c8c21af06a15 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:27:07.474754 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760e770c-0ef9-4321-90a4-c8c21af06a15 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:27:07.475054 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760e770c-0ef9-4321-90a4-c8c21af06a15 None admin] PATH_INFO: `/v3/projects` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:27:07.475929 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-760e770c-0ef9-4321-90a4-c8c21af06a15 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:27:07.476285 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-760e770c-0ef9-4321-90a4-c8c21af06a15 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:27:07.477934 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-760e770c-0ef9-4321-90a4-c8c21af06a15 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:27:07.492345 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 617/1218] 10.222.0.33 () {56 vars in 1148 bytes} [Mon Jul 25 05:27:07 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:27:07.498406 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da7bc4d-7c6e-47d1-9cdc-3ee887bfdca5 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:27:07.614300 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da7bc4d-7c6e-47d1-9cdc-3ee887bfdca5 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:27:07.646776 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:27:07.656502 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e00390e-746e-4e2c-90e5-5251c57a4ad2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:27:07.657192 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e00390e-746e-4e2c-90e5-5251c57a4ad2 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:27:07.657467 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e00390e-746e-4e2c-90e5-5251c57a4ad2 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:27:07.657743 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e00390e-746e-4e2c-90e5-5251c57a4ad2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:27:07.658581 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e00390e-746e-4e2c-90e5-5251c57a4ad2 None admin] RBAC: Adding query filter params (role.id=200e95cafbc54269a51756ddd406082f, scope.project.id=92afba266cce4d1bad4134c351e74be3, user.id=317790016e7047fe96af73925de5d872) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:27:07.658903 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e00390e-746e-4e2c-90e5-5251c57a4ad2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:27:07.660310 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e00390e-746e-4e2c-90e5-5251c57a4ad2 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:27:07.666581 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 602/1219] 10.222.0.33 () {56 vars in 1374 bytes} [Mon Jul 25 05:27:07 2022] GET /identity/v3/role_assignments?user.id=317790016e7047fe96af73925de5d872&scope.project.id=92afba266cce4d1bad4134c351e74be3&role.id=200e95cafbc54269a51756ddd406082f => generated 259 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:27:09.831392 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c9a3e6-a287-4515-b110-15743d141833 None None] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:27:09.832123 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c9a3e6-a287-4515-b110-15743d141833 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:27:09.832123 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c9a3e6-a287-4515-b110-15743d141833 None None] PATH_INFO: `/` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:27:09.833416 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 618/1220] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jul 25 05:27:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:27:09.839686 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43188d1c-ddb6-44c1-9861-fe8342c82414 None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:27:09.840326 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43188d1c-ddb6-44c1-9861-fe8342c82414 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:27:09.840326 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43188d1c-ddb6-44c1-9861-fe8342c82414 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:27:09.858045 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-43188d1c-ddb6-44c1-9861-fe8342c82414 None None] MFA Rules not processed for user `1eae540242d94c10b0ad33960a855f08`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:27:09.859450 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-43188d1c-ddb6-44c1-9861-fe8342c82414 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:27:09.929032 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 603/1221] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jul 25 05:27:09 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 25 05:27:09.955930 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c9a3e6-a287-4515-b110-15743d141833 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:27:10.066755 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c9a3e6-a287-4515-b110-15743d141833 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:27:10.102805 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0efdbb0b-dfce-49c2-91c6-9ecb97cef274 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:27:10.103725 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0efdbb0b-dfce-49c2-91c6-9ecb97cef274 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:27:10.104009 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0efdbb0b-dfce-49c2-91c6-9ecb97cef274 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:27:10.104285 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0efdbb0b-dfce-49c2-91c6-9ecb97cef274 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:27:10.116469 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0efdbb0b-dfce-49c2-91c6-9ecb97cef274 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 05:27:10.125929 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application [None req-0efdbb0b-dfce-49c2-91c6-9ecb97cef274 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 25 05:27:10.125929 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:27:10.125929 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:27:10.125929 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:27:10.125929 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:27:10.125929 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:27:10.125929 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:27:10.125929 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:27:10.125929 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:27:10.125929 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:27:10.125929 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:27:10.125929 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:27:10.125929 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 05:27:10.125929 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 05:27:10.125929 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 05:27:10.125929 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 05:27:10.127251 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 05:27:10.127251 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 05:27:10.127251 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 05:27:10.127251 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 05:27:10.127251 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:27:10.127251 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:27:10.127251 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:27:10.127251 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:27:10.127251 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:27:10.127251 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:27:10.127251 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:27:10.127251 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application with Lock( Jul 25 05:27:10.127251 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:27:10.127251 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:27:10.127251 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:27:10.127251 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:27:10.128131 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:27:10.128131 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:27:10.128131 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:27:10.128131 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:27:10.128131 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 05:27:10.128131 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 05:27:10.128131 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 05:27:10.128131 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 05:27:10.128131 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 05:27:10.128131 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 05:27:10.128131 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 25 05:27:10.128131 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Jul 25 05:27:10.128131 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 619/1222] 10.222.0.33 () {56 vars in 1102 bytes} [Mon Jul 25 05:27:09 2022] GET /identity/v3/roles/service => generated 85 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:27:10.134037 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43188d1c-ddb6-44c1-9861-fe8342c82414 None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:27:10.239249 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43188d1c-ddb6-44c1-9861-fe8342c82414 None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:27:10.265499 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:27:10.277579 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d4ce75e-8cd9-4269-8400-1f9e30514f38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:27:10.278257 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4ce75e-8cd9-4269-8400-1f9e30514f38 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:27:10.278546 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4ce75e-8cd9-4269-8400-1f9e30514f38 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:27:10.278752 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4ce75e-8cd9-4269-8400-1f9e30514f38 None admin] PATH_INFO: `/v3/roles` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:27:10.279597 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d4ce75e-8cd9-4269-8400-1f9e30514f38 None admin] RBAC: Adding query filter params (name=service) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:27:10.279956 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d4ce75e-8cd9-4269-8400-1f9e30514f38 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:27:10.281294 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d4ce75e-8cd9-4269-8400-1f9e30514f38 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:27:10.290164 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 604/1223] 10.222.0.33 () {56 vars in 1103 bytes} [Mon Jul 25 05:27:10 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:27:10.297124 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0efdbb0b-dfce-49c2-91c6-9ecb97cef274 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:27:10.346385 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:27:10.407867 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0efdbb0b-dfce-49c2-91c6-9ecb97cef274 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:27:10.444515 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-821e9e4c-f75c-471c-b011-474bc0c466c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:27:10.445332 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821e9e4c-f75c-471c-b011-474bc0c466c7 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:27:10.445616 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821e9e4c-f75c-471c-b011-474bc0c466c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:27:10.445891 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821e9e4c-f75c-471c-b011-474bc0c466c7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:27:10.457495 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-821e9e4c-f75c-471c-b011-474bc0c466c7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:27:10.459221 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-821e9e4c-f75c-471c-b011-474bc0c466c7 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:27:10.476156 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application [None req-821e9e4c-f75c-471c-b011-474bc0c466c7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 05:27:10.476156 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:27:10.476156 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:27:10.476156 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:27:10.476156 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:27:10.476156 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:27:10.476156 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:27:10.476156 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:27:10.476156 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:27:10.476156 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:27:10.476156 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:27:10.476156 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:27:10.476156 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 05:27:10.476156 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 05:27:10.476156 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 05:27:10.477302 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 05:27:10.477302 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:27:10.477302 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:27:10.477302 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:27:10.477302 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:27:10.477302 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:27:10.477302 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:27:10.477302 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:27:10.477302 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application with Lock( Jul 25 05:27:10.477302 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:27:10.477302 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:27:10.477302 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:27:10.477302 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:27:10.477302 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:27:10.477302 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:27:10.477302 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:27:10.477302 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:27:10.478245 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 05:27:10.478245 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 05:27:10.478245 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 05:27:10.478245 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Jul 25 05:27:10.478245 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 620/1224] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Jul 25 05:27:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:27:10.484756 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d4ce75e-8cd9-4269-8400-1f9e30514f38 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:27:10.605573 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d4ce75e-8cd9-4269-8400-1f9e30514f38 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:27:10.639610 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb81e62e-d34c-47e4-91b9-70edbbacc4d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:27:10.640277 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb81e62e-d34c-47e4-91b9-70edbbacc4d6 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:27:10.640565 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb81e62e-d34c-47e4-91b9-70edbbacc4d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:27:10.640871 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb81e62e-d34c-47e4-91b9-70edbbacc4d6 None admin] PATH_INFO: `/v3/domains` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:27:10.641598 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb81e62e-d34c-47e4-91b9-70edbbacc4d6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:27:10.641909 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb81e62e-d34c-47e4-91b9-70edbbacc4d6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:27:10.643791 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb81e62e-d34c-47e4-91b9-70edbbacc4d6 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:27:10.655996 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 605/1225] 10.222.0.33 () {56 vars in 1109 bytes} [Mon Jul 25 05:27:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:27:10.662676 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-821e9e4c-f75c-471c-b011-474bc0c466c7 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:27:10.760518 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:27:10.768820 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-821e9e4c-f75c-471c-b011-474bc0c466c7 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:27:10.809981 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2efdd094-a842-44fa-8c32-8502ecaa45f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:27:10.811088 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2efdd094-a842-44fa-8c32-8502ecaa45f9 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:27:10.811520 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2efdd094-a842-44fa-8c32-8502ecaa45f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:27:10.811806 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2efdd094-a842-44fa-8c32-8502ecaa45f9 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:27:10.821920 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2efdd094-a842-44fa-8c32-8502ecaa45f9 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:27:10.823351 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2efdd094-a842-44fa-8c32-8502ecaa45f9 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:27:10.838350 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application [None req-2efdd094-a842-44fa-8c32-8502ecaa45f9 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 25 05:27:10.838350 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:27:10.838350 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:27:10.838350 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:27:10.838350 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:27:10.838350 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:27:10.838350 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:27:10.838350 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:27:10.838350 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:27:10.838350 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:27:10.838350 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:27:10.838350 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:27:10.838350 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 25 05:27:10.838350 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 25 05:27:10.838350 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 25 05:27:10.838350 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 25 05:27:10.840533 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:27:10.840533 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:27:10.840533 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 25 05:27:10.840533 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 05:27:10.840533 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 25 05:27:10.840533 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 05:27:10.840533 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:27:10.840533 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:27:10.840533 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:27:10.840533 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:27:10.840533 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:27:10.840533 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application with Lock( Jul 25 05:27:10.840533 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:27:10.840533 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:27:10.840533 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:27:10.840533 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:27:10.840533 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:27:10.840533 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:27:10.841530 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:27:10.841530 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:27:10.841530 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 25 05:27:10.841530 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 25 05:27:10.841530 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 25 05:27:10.841530 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 25 05:27:10.841530 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 25 05:27:10.841530 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 25 05:27:10.841530 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 25 05:27:10.841530 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Jul 25 05:27:10.841530 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 621/1226] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Jul 25 05:27:10 2022] GET /identity/v3/users/placement => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:27:10.846348 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb81e62e-d34c-47e4-91b9-70edbbacc4d6 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:27:10.898758 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:27:10.961520 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb81e62e-d34c-47e4-91b9-70edbbacc4d6 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:27:11.006774 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc468c52-d3c6-4f57-9e10-425b5290f2f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:27:11.007450 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc468c52-d3c6-4f57-9e10-425b5290f2f0 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:27:11.007736 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc468c52-d3c6-4f57-9e10-425b5290f2f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:27:11.008011 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc468c52-d3c6-4f57-9e10-425b5290f2f0 None admin] PATH_INFO: `/v3/users` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:27:11.010425 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc468c52-d3c6-4f57-9e10-425b5290f2f0 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:27:11.010425 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc468c52-d3c6-4f57-9e10-425b5290f2f0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:27:11.010677 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc468c52-d3c6-4f57-9e10-425b5290f2f0 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:27:11.029087 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 606/1227] 10.222.0.33 () {56 vars in 1143 bytes} [Mon Jul 25 05:27:10 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:27:11.034972 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2efdd094-a842-44fa-8c32-8502ecaa45f9 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:27:11.147630 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2efdd094-a842-44fa-8c32-8502ecaa45f9 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:27:11.185166 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c85e4a1f-d2ab-46d3-a60b-8c66cfd6a689 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:27:11.185896 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85e4a1f-d2ab-46d3-a60b-8c66cfd6a689 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:27:11.186277 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85e4a1f-d2ab-46d3-a60b-8c66cfd6a689 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:27:11.186677 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85e4a1f-d2ab-46d3-a60b-8c66cfd6a689 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:27:11.199629 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c85e4a1f-d2ab-46d3-a60b-8c66cfd6a689 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:27:11.201217 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c85e4a1f-d2ab-46d3-a60b-8c66cfd6a689 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:27:11.219496 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application [None req-c85e4a1f-d2ab-46d3-a60b-8c66cfd6a689 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 05:27:11.219496 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:27:11.219496 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:27:11.219496 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:27:11.219496 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:27:11.219496 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:27:11.219496 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:27:11.219496 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:27:11.219496 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:27:11.219496 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:27:11.219496 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:27:11.219496 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:27:11.219496 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 05:27:11.219496 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 05:27:11.219496 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 05:27:11.220647 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 05:27:11.220647 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:27:11.220647 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:27:11.220647 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:27:11.220647 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:27:11.220647 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:27:11.220647 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:27:11.220647 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:27:11.220647 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application with Lock( Jul 25 05:27:11.220647 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:27:11.220647 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:27:11.220647 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:27:11.220647 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:27:11.220647 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:27:11.220647 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:27:11.220647 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:27:11.220647 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:27:11.221732 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 05:27:11.221732 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 05:27:11.221732 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 05:27:11.221732 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Jul 25 05:27:11.221732 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 622/1228] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Jul 25 05:27:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:27:11.226582 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc468c52-d3c6-4f57-9e10-425b5290f2f0 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:27:11.346520 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc468c52-d3c6-4f57-9e10-425b5290f2f0 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:27:11.348919 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:27:11.391350 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc54673-db0a-4338-b58e-18c20a480453 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:27:11.392320 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc54673-db0a-4338-b58e-18c20a480453 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:27:11.392609 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc54673-db0a-4338-b58e-18c20a480453 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:27:11.392949 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc54673-db0a-4338-b58e-18c20a480453 None admin] PATH_INFO: `/v3/domains` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:27:11.393863 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cc54673-db0a-4338-b58e-18c20a480453 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:27:11.394425 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cc54673-db0a-4338-b58e-18c20a480453 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:27:11.395881 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cc54673-db0a-4338-b58e-18c20a480453 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:27:11.408984 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 607/1229] 10.222.0.33 () {56 vars in 1109 bytes} [Mon Jul 25 05:27:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:27:11.414492 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c85e4a1f-d2ab-46d3-a60b-8c66cfd6a689 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:27:11.424503 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:27:11.527884 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c85e4a1f-d2ab-46d3-a60b-8c66cfd6a689 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:27:11.566246 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d1017b-a11b-4903-b720-c4d461771b37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:27:11.567190 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d1017b-a11b-4903-b720-c4d461771b37 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:27:11.567512 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d1017b-a11b-4903-b720-c4d461771b37 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:27:11.567788 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d1017b-a11b-4903-b720-c4d461771b37 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:27:11.576517 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71d1017b-a11b-4903-b720-c4d461771b37 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:27:11.578090 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71d1017b-a11b-4903-b720-c4d461771b37 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:27:11.594437 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application [None req-71d1017b-a11b-4903-b720-c4d461771b37 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 05:27:11.594437 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:27:11.594437 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:27:11.594437 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:27:11.594437 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:27:11.594437 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:27:11.594437 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:27:11.594437 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:27:11.594437 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:27:11.594437 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:27:11.594437 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:27:11.594437 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:27:11.594437 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 25 05:27:11.594437 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 25 05:27:11.594437 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 25 05:27:11.596321 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 25 05:27:11.596321 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:27:11.596321 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:27:11.596321 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:27:11.596321 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:27:11.596321 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:27:11.596321 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:27:11.596321 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:27:11.596321 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application with Lock( Jul 25 05:27:11.596321 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:27:11.596321 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:27:11.596321 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:27:11.596321 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:27:11.596321 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:27:11.596321 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:27:11.596321 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:27:11.596321 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:27:11.597707 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 25 05:27:11.597707 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 25 05:27:11.597707 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 25 05:27:11.597707 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 25 05:27:11.597707 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 25 05:27:11.597707 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 25 05:27:11.597707 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 05:27:11.597707 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Jul 25 05:27:11.597707 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 623/1230] 10.222.0.33 () {56 vars in 1111 bytes} [Mon Jul 25 05:27:11 2022] GET /identity/v3/projects/service => generated 88 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:27:11.601725 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc54673-db0a-4338-b58e-18c20a480453 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:27:11.708753 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc54673-db0a-4338-b58e-18c20a480453 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:27:11.744799 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acefb9dc-d216-4b19-8f60-4059a2ec38f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:27:11.745564 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acefb9dc-d216-4b19-8f60-4059a2ec38f4 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:27:11.745853 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acefb9dc-d216-4b19-8f60-4059a2ec38f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:27:11.746135 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acefb9dc-d216-4b19-8f60-4059a2ec38f4 None admin] PATH_INFO: `/v3/projects` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:27:11.746988 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acefb9dc-d216-4b19-8f60-4059a2ec38f4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:27:11.747232 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acefb9dc-d216-4b19-8f60-4059a2ec38f4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:27:11.748675 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acefb9dc-d216-4b19-8f60-4059a2ec38f4 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:27:11.761495 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 608/1231] 10.222.0.33 () {56 vars in 1148 bytes} [Mon Jul 25 05:27:11 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:27:11.767498 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d1017b-a11b-4903-b720-c4d461771b37 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:27:11.842299 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:27:11.881199 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d1017b-a11b-4903-b720-c4d461771b37 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:27:11.918505 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced867e7-6bf3-4977-ba8a-b80d11702be7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:27:11.919508 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced867e7-6bf3-4977-ba8a-b80d11702be7 None admin] REQUEST_METHOD: `PUT` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:27:11.919836 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced867e7-6bf3-4977-ba8a-b80d11702be7 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:27:11.920150 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced867e7-6bf3-4977-ba8a-b80d11702be7 None admin] PATH_INFO: `/v3/projects/92afba266cce4d1bad4134c351e74be3/users/317790016e7047fe96af73925de5d872/roles/200e95cafbc54269a51756ddd406082f` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:27:11.937354 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ced867e7-6bf3-4977-ba8a-b80d11702be7 None admin] RBAC: Authorizing `identity:create_grant(project_id=92afba266cce4d1bad4134c351e74be3, user_id=317790016e7047fe96af73925de5d872, role_id=200e95cafbc54269a51756ddd406082f)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:27:11.938842 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ced867e7-6bf3-4977-ba8a-b80d11702be7 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:27:11.953024 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 624/1232] 10.222.0.33 () {58 vars in 1439 bytes} [Mon Jul 25 05:27:11 2022] PUT /identity/v3/projects/92afba266cce4d1bad4134c351e74be3/users/317790016e7047fe96af73925de5d872/roles/200e95cafbc54269a51756ddd406082f => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 05:27:14.149584 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08fdc0ef-ba66-4f14-b67d-0db456b97692 None None] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:27:14.150102 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08fdc0ef-ba66-4f14-b67d-0db456b97692 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:27:14.150102 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08fdc0ef-ba66-4f14-b67d-0db456b97692 None None] PATH_INFO: `/` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:27:14.151584 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 609/1233] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jul 25 05:27:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:27:14.157146 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc781f7-a25a-43e9-8594-dcd5a15e0420 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:27:14.157584 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc781f7-a25a-43e9-8594-dcd5a15e0420 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:27:14.157715 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc781f7-a25a-43e9-8594-dcd5a15e0420 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:27:14.176336 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-adc781f7-a25a-43e9-8594-dcd5a15e0420 None None] MFA Rules not processed for user `1eae540242d94c10b0ad33960a855f08`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:27:14.177571 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-adc781f7-a25a-43e9-8594-dcd5a15e0420 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:27:14.248589 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 625/1234] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jul 25 05:27:14 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 25 05:27:14.273977 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08fdc0ef-ba66-4f14-b67d-0db456b97692 None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:27:14.302564 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:27:14.395134 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08fdc0ef-ba66-4f14-b67d-0db456b97692 None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:27:14.435903 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0bd39c2-1056-44d9-b0bc-2ff97b5ec1d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:27:14.436868 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0bd39c2-1056-44d9-b0bc-2ff97b5ec1d4 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:27:14.437206 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0bd39c2-1056-44d9-b0bc-2ff97b5ec1d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:27:14.437432 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0bd39c2-1056-44d9-b0bc-2ff97b5ec1d4 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:27:14.449934 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0bd39c2-1056-44d9-b0bc-2ff97b5ec1d4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 05:27:14.458513 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application [None req-e0bd39c2-1056-44d9-b0bc-2ff97b5ec1d4 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 25 05:27:14.458513 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:27:14.458513 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:27:14.458513 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:27:14.458513 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:27:14.458513 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:27:14.458513 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:27:14.458513 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:27:14.458513 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:27:14.458513 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:27:14.458513 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:27:14.458513 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:27:14.458513 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 05:27:14.458513 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 05:27:14.458513 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 05:27:14.458513 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 05:27:14.459583 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 05:27:14.459583 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 05:27:14.459583 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 05:27:14.459583 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 05:27:14.459583 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:27:14.459583 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:27:14.459583 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:27:14.459583 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:27:14.459583 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:27:14.459583 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:27:14.459583 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:27:14.459583 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application with Lock( Jul 25 05:27:14.459583 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:27:14.459583 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:27:14.459583 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:27:14.459583 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:27:14.460496 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:27:14.460496 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:27:14.460496 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:27:14.460496 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:27:14.460496 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 05:27:14.460496 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 05:27:14.460496 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 05:27:14.460496 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 05:27:14.460496 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 05:27:14.460496 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 05:27:14.460496 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 25 05:27:14.460496 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Jul 25 05:27:14.460496 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 610/1235] 10.222.0.33 () {56 vars in 1102 bytes} [Mon Jul 25 05:27:14 2022] GET /identity/v3/roles/service => generated 85 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:27:14.468273 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc781f7-a25a-43e9-8594-dcd5a15e0420 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:27:14.520238 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:27:14.579328 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc781f7-a25a-43e9-8594-dcd5a15e0420 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:27:14.620138 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c3d814-4220-4dce-a9dc-c55a1d09158f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:27:14.620982 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c3d814-4220-4dce-a9dc-c55a1d09158f None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:27:14.621383 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c3d814-4220-4dce-a9dc-c55a1d09158f None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:27:14.621652 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c3d814-4220-4dce-a9dc-c55a1d09158f None admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:27:14.622527 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43c3d814-4220-4dce-a9dc-c55a1d09158f None admin] RBAC: Adding query filter params (name=service) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:27:14.622842 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43c3d814-4220-4dce-a9dc-c55a1d09158f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:27:14.624422 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43c3d814-4220-4dce-a9dc-c55a1d09158f None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:27:14.633809 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 626/1236] 10.222.0.33 () {56 vars in 1103 bytes} [Mon Jul 25 05:27:14 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:27:14.640505 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0bd39c2-1056-44d9-b0bc-2ff97b5ec1d4 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:27:14.734815 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:27:14.752099 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0bd39c2-1056-44d9-b0bc-2ff97b5ec1d4 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:27:14.788258 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b668e1-c9fd-46e5-866d-6816ca4baa7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:27:14.789092 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b668e1-c9fd-46e5-866d-6816ca4baa7d None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:27:14.789376 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b668e1-c9fd-46e5-866d-6816ca4baa7d None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:27:14.789717 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b668e1-c9fd-46e5-866d-6816ca4baa7d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:27:14.803266 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7b668e1-c9fd-46e5-866d-6816ca4baa7d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:27:14.804814 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7b668e1-c9fd-46e5-866d-6816ca4baa7d None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:27:14.823696 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application [None req-f7b668e1-c9fd-46e5-866d-6816ca4baa7d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 05:27:14.823696 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:27:14.823696 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:27:14.823696 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:27:14.823696 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:27:14.823696 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:27:14.823696 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:27:14.823696 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:27:14.823696 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:27:14.823696 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:27:14.823696 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:27:14.823696 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:27:14.823696 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 05:27:14.823696 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 05:27:14.823696 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 05:27:14.824856 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 05:27:14.824856 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:27:14.824856 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:27:14.824856 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:27:14.824856 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:27:14.824856 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:27:14.824856 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:27:14.824856 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:27:14.824856 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application with Lock( Jul 25 05:27:14.824856 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:27:14.824856 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:27:14.824856 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:27:14.824856 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:27:14.824856 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:27:14.824856 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:27:14.824856 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:27:14.824856 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:27:14.825790 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 05:27:14.825790 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 05:27:14.825790 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 05:27:14.825790 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Jul 25 05:27:14.825790 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 611/1237] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Jul 25 05:27:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 25 05:27:14.831120 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c3d814-4220-4dce-a9dc-c55a1d09158f None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:27:14.940963 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c3d814-4220-4dce-a9dc-c55a1d09158f None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:27:14.952079 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:27:14.981624 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c6a6131-e53c-48b4-a82a-091ea69f1f92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:27:14.982384 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6a6131-e53c-48b4-a82a-091ea69f1f92 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:27:14.982668 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6a6131-e53c-48b4-a82a-091ea69f1f92 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:27:14.982983 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6a6131-e53c-48b4-a82a-091ea69f1f92 None admin] PATH_INFO: `/v3/domains` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:27:14.983814 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c6a6131-e53c-48b4-a82a-091ea69f1f92 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:27:14.984155 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c6a6131-e53c-48b4-a82a-091ea69f1f92 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:27:14.986031 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c6a6131-e53c-48b4-a82a-091ea69f1f92 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:27:14.998682 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 627/1238] 10.222.0.33 () {56 vars in 1109 bytes} [Mon Jul 25 05:27:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:27:15.004839 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b668e1-c9fd-46e5-866d-6816ca4baa7d None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:27:15.119433 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b668e1-c9fd-46e5-866d-6816ca4baa7d None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:27:15.156667 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4061a0c2-10d9-4093-8971-8de417e98595 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:27:15.157496 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4061a0c2-10d9-4093-8971-8de417e98595 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:27:15.157777 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4061a0c2-10d9-4093-8971-8de417e98595 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:27:15.158117 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4061a0c2-10d9-4093-8971-8de417e98595 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:27:15.161430 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:27:15.169833 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4061a0c2-10d9-4093-8971-8de417e98595 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:27:15.171875 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4061a0c2-10d9-4093-8971-8de417e98595 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:27:15.189162 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application [None req-4061a0c2-10d9-4093-8971-8de417e98595 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 25 05:27:15.189162 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:27:15.189162 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:27:15.189162 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:27:15.189162 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:27:15.189162 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:27:15.189162 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:27:15.189162 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:27:15.189162 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:27:15.189162 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:27:15.189162 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:27:15.189162 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:27:15.189162 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 25 05:27:15.189162 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 25 05:27:15.189162 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 25 05:27:15.189162 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 25 05:27:15.190230 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:27:15.190230 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:27:15.190230 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 25 05:27:15.190230 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 05:27:15.190230 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 25 05:27:15.190230 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 05:27:15.190230 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:27:15.190230 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:27:15.190230 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:27:15.190230 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:27:15.190230 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:27:15.190230 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application with Lock( Jul 25 05:27:15.190230 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:27:15.190230 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:27:15.190230 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:27:15.190230 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:27:15.190230 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:27:15.190230 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:27:15.191207 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:27:15.191207 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:27:15.191207 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 25 05:27:15.191207 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 25 05:27:15.191207 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 25 05:27:15.191207 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 25 05:27:15.191207 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 25 05:27:15.191207 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 25 05:27:15.191207 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 25 05:27:15.191207 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Jul 25 05:27:15.191207 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 612/1239] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Jul 25 05:27:15 2022] GET /identity/v3/users/placement => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:27:15.197080 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c6a6131-e53c-48b4-a82a-091ea69f1f92 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:27:15.314378 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c6a6131-e53c-48b4-a82a-091ea69f1f92 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:27:15.354229 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c58c3fcf-6c5b-4ccc-9367-1d3975804605 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:27:15.354879 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58c3fcf-6c5b-4ccc-9367-1d3975804605 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:27:15.355168 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58c3fcf-6c5b-4ccc-9367-1d3975804605 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:27:15.355449 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58c3fcf-6c5b-4ccc-9367-1d3975804605 None admin] PATH_INFO: `/v3/users` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:27:15.356300 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c58c3fcf-6c5b-4ccc-9367-1d3975804605 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:27:15.356611 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c58c3fcf-6c5b-4ccc-9367-1d3975804605 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:27:15.358092 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c58c3fcf-6c5b-4ccc-9367-1d3975804605 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:27:15.377081 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 628/1240] 10.222.0.33 () {56 vars in 1143 bytes} [Mon Jul 25 05:27:15 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:27:15.382845 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4061a0c2-10d9-4093-8971-8de417e98595 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:27:15.485312 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4061a0c2-10d9-4093-8971-8de417e98595 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:27:15.524243 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b306ac-e196-45cd-a3d0-07beebc061ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:27:15.525164 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b306ac-e196-45cd-a3d0-07beebc061ee None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:27:15.525447 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b306ac-e196-45cd-a3d0-07beebc061ee None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:27:15.525726 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b306ac-e196-45cd-a3d0-07beebc061ee None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:27:15.601405 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:27:15.610044 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8b306ac-e196-45cd-a3d0-07beebc061ee None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:27:15.611710 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8b306ac-e196-45cd-a3d0-07beebc061ee None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:27:15.731970 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 613/1241] 10.222.0.33 () {58 vars in 1282 bytes} [Mon Jul 25 05:27:15 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 350 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 25 05:27:15.738032 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c58c3fcf-6c5b-4ccc-9367-1d3975804605 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:27:15.767978 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:27:15.846176 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c58c3fcf-6c5b-4ccc-9367-1d3975804605 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:27:15.883550 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4becc29d-d90a-43b3-a5c1-77c186d2dfc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:27:15.884280 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4becc29d-d90a-43b3-a5c1-77c186d2dfc5 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:27:15.884568 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4becc29d-d90a-43b3-a5c1-77c186d2dfc5 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:27:15.884887 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4becc29d-d90a-43b3-a5c1-77c186d2dfc5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:27:15.895702 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4becc29d-d90a-43b3-a5c1-77c186d2dfc5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:27:15.897212 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4becc29d-d90a-43b3-a5c1-77c186d2dfc5 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:27:15.915512 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application [None req-4becc29d-d90a-43b3-a5c1-77c186d2dfc5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 05:27:15.915512 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:27:15.915512 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:27:15.915512 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:27:15.915512 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:27:15.915512 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:27:15.915512 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:27:15.915512 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:27:15.915512 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:27:15.915512 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:27:15.915512 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:27:15.915512 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:27:15.915512 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 05:27:15.915512 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 05:27:15.915512 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 05:27:15.916722 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 05:27:15.916722 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:27:15.916722 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:27:15.916722 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:27:15.916722 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:27:15.916722 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:27:15.916722 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:27:15.916722 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:27:15.916722 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application with Lock( Jul 25 05:27:15.916722 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:27:15.916722 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:27:15.916722 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:27:15.916722 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:27:15.916722 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:27:15.916722 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:27:15.916722 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:27:15.916722 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:27:15.918651 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 05:27:15.918651 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 05:27:15.918651 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 05:27:15.918651 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Jul 25 05:27:15.918651 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 629/1242] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Jul 25 05:27:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 25 05:27:15.923543 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b306ac-e196-45cd-a3d0-07beebc061ee None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:27:16.025221 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:27:16.032911 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b306ac-e196-45cd-a3d0-07beebc061ee None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:27:16.070346 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e8c331-aec6-42b2-94d6-ea41d81d817d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:27:16.071057 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e8c331-aec6-42b2-94d6-ea41d81d817d None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:27:16.071460 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e8c331-aec6-42b2-94d6-ea41d81d817d None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:27:16.071777 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e8c331-aec6-42b2-94d6-ea41d81d817d None admin] PATH_INFO: `/v3/domains` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:27:16.072527 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01e8c331-aec6-42b2-94d6-ea41d81d817d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:27:16.072873 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01e8c331-aec6-42b2-94d6-ea41d81d817d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:27:16.074894 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01e8c331-aec6-42b2-94d6-ea41d81d817d None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:27:16.088022 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 614/1243] 10.222.0.33 () {56 vars in 1109 bytes} [Mon Jul 25 05:27:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:27:16.094354 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4becc29d-d90a-43b3-a5c1-77c186d2dfc5 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:27:16.181771 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:27:16.203749 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4becc29d-d90a-43b3-a5c1-77c186d2dfc5 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:27:16.238818 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce1115b-9cb7-41ee-a6a5-1a02805ba723 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:27:16.239621 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce1115b-9cb7-41ee-a6a5-1a02805ba723 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:27:16.239932 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce1115b-9cb7-41ee-a6a5-1a02805ba723 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:27:16.240252 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce1115b-9cb7-41ee-a6a5-1a02805ba723 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:27:16.248372 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bce1115b-9cb7-41ee-a6a5-1a02805ba723 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:27:16.249958 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bce1115b-9cb7-41ee-a6a5-1a02805ba723 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:27:16.264256 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application [None req-bce1115b-9cb7-41ee-a6a5-1a02805ba723 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 05:27:16.264256 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:27:16.264256 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:27:16.264256 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:27:16.264256 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:27:16.264256 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:27:16.264256 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:27:16.264256 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:27:16.264256 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:27:16.264256 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:27:16.264256 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:27:16.264256 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:27:16.264256 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 25 05:27:16.264256 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 25 05:27:16.264256 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 25 05:27:16.265713 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 25 05:27:16.265713 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:27:16.265713 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:27:16.265713 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:27:16.265713 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:27:16.265713 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:27:16.265713 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:27:16.265713 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:27:16.265713 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application with Lock( Jul 25 05:27:16.265713 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:27:16.265713 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:27:16.265713 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:27:16.265713 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:27:16.265713 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:27:16.265713 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:27:16.265713 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:27:16.265713 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:27:16.266633 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 25 05:27:16.266633 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 25 05:27:16.266633 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 25 05:27:16.266633 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 25 05:27:16.266633 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 25 05:27:16.266633 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 25 05:27:16.266633 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 05:27:16.266633 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Jul 25 05:27:16.266633 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 630/1244] 10.222.0.33 () {56 vars in 1111 bytes} [Mon Jul 25 05:27:16 2022] GET /identity/v3/projects/service => generated 88 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:27:16.271518 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e8c331-aec6-42b2-94d6-ea41d81d817d None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:27:16.387127 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e8c331-aec6-42b2-94d6-ea41d81d817d None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:27:16.427582 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-422f6003-f239-4ba7-9402-b0d167bdeedb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:27:16.428339 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422f6003-f239-4ba7-9402-b0d167bdeedb None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:27:16.428702 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422f6003-f239-4ba7-9402-b0d167bdeedb None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:27:16.429020 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422f6003-f239-4ba7-9402-b0d167bdeedb None admin] PATH_INFO: `/v3/projects` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:27:16.429876 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-422f6003-f239-4ba7-9402-b0d167bdeedb None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:27:16.430156 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-422f6003-f239-4ba7-9402-b0d167bdeedb None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:27:16.431669 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-422f6003-f239-4ba7-9402-b0d167bdeedb None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:27:16.449842 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 615/1245] 10.222.0.33 () {56 vars in 1148 bytes} [Mon Jul 25 05:27:16 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:27:16.452957 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce1115b-9cb7-41ee-a6a5-1a02805ba723 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:27:16.560457 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce1115b-9cb7-41ee-a6a5-1a02805ba723 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:27:16.602703 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9050d2f1-abaa-476c-9912-78e302d9c324 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:27:16.603559 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9050d2f1-abaa-476c-9912-78e302d9c324 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:27:16.603925 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9050d2f1-abaa-476c-9912-78e302d9c324 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:27:16.604234 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9050d2f1-abaa-476c-9912-78e302d9c324 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:27:16.605995 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9050d2f1-abaa-476c-9912-78e302d9c324 None admin] RBAC: Adding query filter params (role.id=200e95cafbc54269a51756ddd406082f, scope.project.id=92afba266cce4d1bad4134c351e74be3, user.id=317790016e7047fe96af73925de5d872) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:27:16.606394 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9050d2f1-abaa-476c-9912-78e302d9c324 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:27:16.607953 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9050d2f1-abaa-476c-9912-78e302d9c324 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:27:16.614730 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 631/1246] 10.222.0.33 () {56 vars in 1374 bytes} [Mon Jul 25 05:27:16 2022] GET /identity/v3/role_assignments?user.id=317790016e7047fe96af73925de5d872&scope.project.id=92afba266cce4d1bad4134c351e74be3&role.id=200e95cafbc54269a51756ddd406082f => generated 608 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:27:18.836263 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cbba2d6-453d-4d5e-82be-90f7198eeb67 None None] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:27:18.836813 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cbba2d6-453d-4d5e-82be-90f7198eeb67 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:27:18.836964 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cbba2d6-453d-4d5e-82be-90f7198eeb67 None None] PATH_INFO: `/` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:27:18.838245 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 616/1247] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jul 25 05:27:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:27:18.844620 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f5c80b-d47e-4d27-9309-1d5d3fdf77dc None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:27:18.845176 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f5c80b-d47e-4d27-9309-1d5d3fdf77dc None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:27:18.845289 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f5c80b-d47e-4d27-9309-1d5d3fdf77dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:27:18.850689 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:27:18.864293 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-35f5c80b-d47e-4d27-9309-1d5d3fdf77dc None None] MFA Rules not processed for user `1eae540242d94c10b0ad33960a855f08`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:27:18.865532 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-35f5c80b-d47e-4d27-9309-1d5d3fdf77dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:27:18.938851 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 632/1248] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jul 25 05:27:18 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 25 05:27:18.962515 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cbba2d6-453d-4d5e-82be-90f7198eeb67 None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:27:18.984092 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:27:19.078465 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cbba2d6-453d-4d5e-82be-90f7198eeb67 None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:27:19.116201 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3afe676-e6c1-44ad-9038-4fd49f9bdfbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:27:19.117136 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3afe676-e6c1-44ad-9038-4fd49f9bdfbd None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:27:19.117461 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3afe676-e6c1-44ad-9038-4fd49f9bdfbd None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:27:19.117771 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3afe676-e6c1-44ad-9038-4fd49f9bdfbd None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:27:19.131120 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3afe676-e6c1-44ad-9038-4fd49f9bdfbd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 05:27:19.139766 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application [None req-e3afe676-e6c1-44ad-9038-4fd49f9bdfbd None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 05:27:19.139766 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:27:19.139766 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:27:19.139766 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:27:19.139766 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:27:19.139766 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:27:19.139766 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:27:19.139766 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:27:19.139766 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:27:19.139766 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:27:19.139766 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:27:19.139766 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:27:19.139766 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 05:27:19.139766 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 05:27:19.139766 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 05:27:19.139766 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 05:27:19.140936 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 05:27:19.140936 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 05:27:19.140936 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 05:27:19.140936 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 05:27:19.140936 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:27:19.140936 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:27:19.140936 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:27:19.140936 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:27:19.140936 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:27:19.140936 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:27:19.140936 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:27:19.140936 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application with Lock( Jul 25 05:27:19.140936 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:27:19.140936 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:27:19.140936 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:27:19.140936 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:27:19.141846 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:27:19.141846 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:27:19.141846 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:27:19.141846 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:27:19.141846 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 05:27:19.141846 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 05:27:19.141846 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 05:27:19.141846 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 05:27:19.141846 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 05:27:19.141846 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 05:27:19.141846 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 05:27:19.141846 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Jul 25 05:27:19.141846 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 617/1249] 10.222.0.33 () {56 vars in 1096 bytes} [Mon Jul 25 05:27:18 2022] GET /identity/v3/roles/admin => generated 83 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:27:19.147144 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f5c80b-d47e-4d27-9309-1d5d3fdf77dc None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:27:19.258438 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f5c80b-d47e-4d27-9309-1d5d3fdf77dc None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:27:19.290761 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:27:19.296417 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d12b6fac-a590-4a92-9054-7b84497c610b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:27:19.297166 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12b6fac-a590-4a92-9054-7b84497c610b None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:27:19.297453 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12b6fac-a590-4a92-9054-7b84497c610b None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:27:19.297728 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12b6fac-a590-4a92-9054-7b84497c610b None admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:27:19.298484 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d12b6fac-a590-4a92-9054-7b84497c610b None admin] RBAC: Adding query filter params (name=admin) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:27:19.298849 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d12b6fac-a590-4a92-9054-7b84497c610b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:27:19.300190 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d12b6fac-a590-4a92-9054-7b84497c610b None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:27:19.308767 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 633/1250] 10.222.0.33 () {56 vars in 1099 bytes} [Mon Jul 25 05:27:19 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 05:27:19.315211 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3afe676-e6c1-44ad-9038-4fd49f9bdfbd None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:27:19.401566 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:27:19.430662 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3afe676-e6c1-44ad-9038-4fd49f9bdfbd None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:27:19.468169 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-480452e4-ed90-4e58-887f-7b42f379c4c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:27:19.469010 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480452e4-ed90-4e58-887f-7b42f379c4c0 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:27:19.469292 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480452e4-ed90-4e58-887f-7b42f379c4c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:27:19.469566 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480452e4-ed90-4e58-887f-7b42f379c4c0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:27:19.483359 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-480452e4-ed90-4e58-887f-7b42f379c4c0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:27:19.485059 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-480452e4-ed90-4e58-887f-7b42f379c4c0 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:27:19.503624 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application [None req-480452e4-ed90-4e58-887f-7b42f379c4c0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 05:27:19.503624 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:27:19.503624 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:27:19.503624 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:27:19.503624 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:27:19.503624 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:27:19.503624 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:27:19.503624 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:27:19.503624 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:27:19.503624 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:27:19.503624 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:27:19.503624 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:27:19.503624 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 05:27:19.503624 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 05:27:19.503624 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 05:27:19.506848 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 05:27:19.506848 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:27:19.506848 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:27:19.506848 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:27:19.506848 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:27:19.506848 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:27:19.506848 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:27:19.506848 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:27:19.506848 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application with Lock( Jul 25 05:27:19.506848 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:27:19.506848 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:27:19.506848 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:27:19.506848 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:27:19.506848 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:27:19.506848 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:27:19.506848 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:27:19.506848 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:27:19.508020 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 05:27:19.508020 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 05:27:19.508020 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 05:27:19.508020 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Jul 25 05:27:19.508020 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 618/1251] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Jul 25 05:27:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:27:19.510686 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d12b6fac-a590-4a92-9054-7b84497c610b None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:27:19.617817 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d12b6fac-a590-4a92-9054-7b84497c610b None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:27:19.657210 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3439ddf6-ee4b-403c-8d22-9dc63b8652b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:27:19.657827 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3439ddf6-ee4b-403c-8d22-9dc63b8652b9 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:27:19.658114 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3439ddf6-ee4b-403c-8d22-9dc63b8652b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:27:19.658396 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3439ddf6-ee4b-403c-8d22-9dc63b8652b9 None admin] PATH_INFO: `/v3/domains` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:27:19.659138 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3439ddf6-ee4b-403c-8d22-9dc63b8652b9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:27:19.659447 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3439ddf6-ee4b-403c-8d22-9dc63b8652b9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:27:19.660825 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3439ddf6-ee4b-403c-8d22-9dc63b8652b9 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:27:19.675372 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 634/1252] 10.222.0.33 () {56 vars in 1109 bytes} [Mon Jul 25 05:27:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:27:19.681204 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-480452e4-ed90-4e58-887f-7b42f379c4c0 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:27:19.791238 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-480452e4-ed90-4e58-887f-7b42f379c4c0 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:27:19.821558 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:27:19.830400 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4dfa40e-3c4f-497b-be36-672cde9ad7ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:27:19.831199 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4dfa40e-3c4f-497b-be36-672cde9ad7ff None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:27:19.831487 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4dfa40e-3c4f-497b-be36-672cde9ad7ff None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:27:19.831765 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4dfa40e-3c4f-497b-be36-672cde9ad7ff None admin] PATH_INFO: `/v3/users/placement` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:27:19.839966 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4dfa40e-3c4f-497b-be36-672cde9ad7ff None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:27:19.841572 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4dfa40e-3c4f-497b-be36-672cde9ad7ff None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:27:19.856670 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application [None req-a4dfa40e-3c4f-497b-be36-672cde9ad7ff None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 25 05:27:19.856670 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:27:19.856670 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:27:19.856670 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:27:19.856670 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:27:19.856670 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:27:19.856670 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:27:19.856670 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:27:19.856670 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:27:19.856670 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:27:19.856670 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:27:19.856670 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:27:19.856670 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 25 05:27:19.856670 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 25 05:27:19.856670 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 25 05:27:19.856670 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 25 05:27:19.858024 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:27:19.858024 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:27:19.858024 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 25 05:27:19.858024 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 05:27:19.858024 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 25 05:27:19.858024 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 05:27:19.858024 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:27:19.858024 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:27:19.858024 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:27:19.858024 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:27:19.858024 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:27:19.858024 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application with Lock( Jul 25 05:27:19.858024 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:27:19.858024 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:27:19.858024 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:27:19.858024 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:27:19.858024 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:27:19.858024 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:27:19.858997 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:27:19.858997 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:27:19.858997 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 25 05:27:19.858997 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 25 05:27:19.858997 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 25 05:27:19.858997 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 25 05:27:19.858997 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 25 05:27:19.858997 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 25 05:27:19.858997 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 25 05:27:19.858997 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Jul 25 05:27:19.858997 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 619/1253] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Jul 25 05:27:19 2022] GET /identity/v3/users/placement => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 25 05:27:19.864599 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3439ddf6-ee4b-403c-8d22-9dc63b8652b9 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:27:19.927178 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:27:19.976330 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3439ddf6-ee4b-403c-8d22-9dc63b8652b9 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:27:20.016675 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e21a42e-e463-417e-98f4-f99d2e80643e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:27:20.017437 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e21a42e-e463-417e-98f4-f99d2e80643e None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:27:20.017721 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e21a42e-e463-417e-98f4-f99d2e80643e None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:27:20.018046 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e21a42e-e463-417e-98f4-f99d2e80643e None admin] PATH_INFO: `/v3/users` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:27:20.018975 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e21a42e-e463-417e-98f4-f99d2e80643e None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:27:20.019326 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e21a42e-e463-417e-98f4-f99d2e80643e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:27:20.020807 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e21a42e-e463-417e-98f4-f99d2e80643e None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:27:20.040036 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 635/1254] 10.222.0.33 () {56 vars in 1143 bytes} [Mon Jul 25 05:27:19 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:27:20.047243 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4dfa40e-3c4f-497b-be36-672cde9ad7ff None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:27:20.160146 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4dfa40e-3c4f-497b-be36-672cde9ad7ff None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:27:20.200558 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f68a09c-6064-4bbb-bedd-3dde9be0ca31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:27:20.201157 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f68a09c-6064-4bbb-bedd-3dde9be0ca31 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:27:20.201525 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f68a09c-6064-4bbb-bedd-3dde9be0ca31 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:27:20.201803 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f68a09c-6064-4bbb-bedd-3dde9be0ca31 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:27:20.258897 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:27:20.273855 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f68a09c-6064-4bbb-bedd-3dde9be0ca31 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:27:20.275489 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f68a09c-6064-4bbb-bedd-3dde9be0ca31 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:27:20.378501 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 620/1255] 10.222.0.33 () {58 vars in 1282 bytes} [Mon Jul 25 05:27:20 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 332 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 25 05:27:20.385236 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e21a42e-e463-417e-98f4-f99d2e80643e None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:27:20.501210 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e21a42e-e463-417e-98f4-f99d2e80643e None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:27:20.515493 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:27:20.538619 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e94047-f672-499f-bd6f-3067bd46654d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:27:20.539417 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e94047-f672-499f-bd6f-3067bd46654d None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:27:20.539707 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e94047-f672-499f-bd6f-3067bd46654d None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:27:20.539991 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e94047-f672-499f-bd6f-3067bd46654d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:27:20.550529 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79e94047-f672-499f-bd6f-3067bd46654d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:27:20.551979 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79e94047-f672-499f-bd6f-3067bd46654d None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:27:20.570485 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application [None req-79e94047-f672-499f-bd6f-3067bd46654d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 05:27:20.570485 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:27:20.570485 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:27:20.570485 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:27:20.570485 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:27:20.570485 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:27:20.570485 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:27:20.570485 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:27:20.570485 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:27:20.570485 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:27:20.570485 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:27:20.570485 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:27:20.570485 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 05:27:20.570485 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 05:27:20.570485 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 05:27:20.571903 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 05:27:20.571903 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:27:20.571903 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:27:20.571903 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:27:20.571903 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:27:20.571903 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:27:20.571903 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:27:20.571903 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:27:20.571903 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application with Lock( Jul 25 05:27:20.571903 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:27:20.571903 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:27:20.571903 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:27:20.571903 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:27:20.571903 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:27:20.571903 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:27:20.571903 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:27:20.571903 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:27:20.573217 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 05:27:20.573217 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 05:27:20.573217 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 05:27:20.573217 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Jul 25 05:27:20.573217 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 636/1256] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Jul 25 05:27:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:27:20.578476 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f68a09c-6064-4bbb-bedd-3dde9be0ca31 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:27:20.671094 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:27:20.683974 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f68a09c-6064-4bbb-bedd-3dde9be0ca31 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:27:20.719913 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-667b3699-e4a3-4d5a-bc47-50cf015b8fa5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:27:20.720619 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667b3699-e4a3-4d5a-bc47-50cf015b8fa5 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:27:20.721048 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667b3699-e4a3-4d5a-bc47-50cf015b8fa5 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:27:20.721248 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667b3699-e4a3-4d5a-bc47-50cf015b8fa5 None admin] PATH_INFO: `/v3/domains` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:27:20.722070 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-667b3699-e4a3-4d5a-bc47-50cf015b8fa5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:27:20.722380 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-667b3699-e4a3-4d5a-bc47-50cf015b8fa5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:27:20.723858 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-667b3699-e4a3-4d5a-bc47-50cf015b8fa5 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:27:20.736068 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 621/1257] 10.222.0.33 () {56 vars in 1109 bytes} [Mon Jul 25 05:27:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:27:20.741874 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e94047-f672-499f-bd6f-3067bd46654d None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:27:20.860278 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e94047-f672-499f-bd6f-3067bd46654d None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:27:20.897568 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd13bb1-ab37-445d-a2c4-31a7a3186cd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:27:20.898357 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd13bb1-ab37-445d-a2c4-31a7a3186cd4 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:27:20.898657 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd13bb1-ab37-445d-a2c4-31a7a3186cd4 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:27:20.898996 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd13bb1-ab37-445d-a2c4-31a7a3186cd4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:27:20.908108 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efd13bb1-ab37-445d-a2c4-31a7a3186cd4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:27:20.910764 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efd13bb1-ab37-445d-a2c4-31a7a3186cd4 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:27:20.925649 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application [None req-efd13bb1-ab37-445d-a2c4-31a7a3186cd4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 05:27:20.925649 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:27:20.925649 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:27:20.925649 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:27:20.925649 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:27:20.925649 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:27:20.925649 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:27:20.925649 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:27:20.925649 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:27:20.925649 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:27:20.925649 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:27:20.925649 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:27:20.925649 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 25 05:27:20.925649 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 25 05:27:20.925649 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 25 05:27:20.929071 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 25 05:27:20.929071 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:27:20.929071 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:27:20.929071 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:27:20.929071 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:27:20.929071 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:27:20.929071 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:27:20.929071 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:27:20.929071 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application with Lock( Jul 25 05:27:20.929071 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:27:20.929071 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:27:20.929071 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:27:20.929071 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:27:20.929071 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:27:20.929071 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:27:20.929071 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:27:20.929071 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:27:20.930218 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 25 05:27:20.930218 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 25 05:27:20.930218 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 25 05:27:20.930218 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 25 05:27:20.930218 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 25 05:27:20.930218 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 25 05:27:20.930218 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 05:27:20.930218 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Jul 25 05:27:20.930218 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 637/1258] 10.222.0.33 () {56 vars in 1111 bytes} [Mon Jul 25 05:27:20 2022] GET /identity/v3/projects/service => generated 88 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 25 05:27:20.933012 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-667b3699-e4a3-4d5a-bc47-50cf015b8fa5 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:27:21.045588 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-667b3699-e4a3-4d5a-bc47-50cf015b8fa5 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:27:21.086021 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb8922d6-8f8b-4f42-900b-7bd4ce609261 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:27:21.086652 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8922d6-8f8b-4f42-900b-7bd4ce609261 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:27:21.087023 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8922d6-8f8b-4f42-900b-7bd4ce609261 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:27:21.087374 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8922d6-8f8b-4f42-900b-7bd4ce609261 None admin] PATH_INFO: `/v3/projects` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:27:21.088261 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb8922d6-8f8b-4f42-900b-7bd4ce609261 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:27:21.088572 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb8922d6-8f8b-4f42-900b-7bd4ce609261 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:27:21.090708 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb8922d6-8f8b-4f42-900b-7bd4ce609261 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:27:21.104691 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 622/1259] 10.222.0.33 () {56 vars in 1148 bytes} [Mon Jul 25 05:27:20 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:27:21.111805 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd13bb1-ab37-445d-a2c4-31a7a3186cd4 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:27:21.148523 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:27:21.230699 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd13bb1-ab37-445d-a2c4-31a7a3186cd4 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:27:21.265723 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18daed2d-3ecd-48e0-b89a-0f2aff9c1dd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:27:21.266403 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18daed2d-3ecd-48e0-b89a-0f2aff9c1dd3 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:27:21.266693 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18daed2d-3ecd-48e0-b89a-0f2aff9c1dd3 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:27:21.266974 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18daed2d-3ecd-48e0-b89a-0f2aff9c1dd3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:27:21.267838 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18daed2d-3ecd-48e0-b89a-0f2aff9c1dd3 None admin] RBAC: Adding query filter params (role.id=39e5c666285a42fe8fc2c3afca6260d1, scope.project.id=92afba266cce4d1bad4134c351e74be3, user.id=317790016e7047fe96af73925de5d872) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:27:21.268152 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18daed2d-3ecd-48e0-b89a-0f2aff9c1dd3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:27:21.269684 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18daed2d-3ecd-48e0-b89a-0f2aff9c1dd3 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:27:21.275938 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 638/1260] 10.222.0.33 () {56 vars in 1374 bytes} [Mon Jul 25 05:27:21 2022] GET /identity/v3/role_assignments?user.id=317790016e7047fe96af73925de5d872&scope.project.id=92afba266cce4d1bad4134c351e74be3&role.id=39e5c666285a42fe8fc2c3afca6260d1 => generated 259 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 05:27:23.321066 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fadc132-aed5-4abd-8135-a61ed5cec731 None None] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:27:23.321597 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fadc132-aed5-4abd-8135-a61ed5cec731 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:27:23.321597 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fadc132-aed5-4abd-8135-a61ed5cec731 None None] PATH_INFO: `/` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:27:23.322975 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 623/1261] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jul 25 05:27:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:27:23.328325 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ce99b1-dd31-4b8d-80db-2121e7af9a1b None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:27:23.328711 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ce99b1-dd31-4b8d-80db-2121e7af9a1b None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:27:23.328927 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ce99b1-dd31-4b8d-80db-2121e7af9a1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:27:23.345795 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-57ce99b1-dd31-4b8d-80db-2121e7af9a1b None None] MFA Rules not processed for user `1eae540242d94c10b0ad33960a855f08`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:27:23.346980 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-57ce99b1-dd31-4b8d-80db-2121e7af9a1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:27:23.411455 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 639/1262] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jul 25 05:27:23 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 25 05:27:23.434953 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fadc132-aed5-4abd-8135-a61ed5cec731 None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:27:23.445971 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:27:23.534077 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fadc132-aed5-4abd-8135-a61ed5cec731 None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:27:23.567540 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8475d43-5054-4f21-a94b-a33fff2ce084 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:27:23.568319 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8475d43-5054-4f21-a94b-a33fff2ce084 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:27:23.568607 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8475d43-5054-4f21-a94b-a33fff2ce084 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:27:23.568922 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8475d43-5054-4f21-a94b-a33fff2ce084 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:27:23.581666 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8475d43-5054-4f21-a94b-a33fff2ce084 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 05:27:23.589377 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application [None req-f8475d43-5054-4f21-a94b-a33fff2ce084 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 05:27:23.589377 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:27:23.589377 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:27:23.589377 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:27:23.589377 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:27:23.589377 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:27:23.589377 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:27:23.589377 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:27:23.589377 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:27:23.589377 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:27:23.589377 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:27:23.589377 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:27:23.589377 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 05:27:23.589377 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 05:27:23.589377 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 05:27:23.589377 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 05:27:23.591229 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 05:27:23.591229 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 05:27:23.591229 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 05:27:23.591229 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 05:27:23.591229 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:27:23.591229 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:27:23.591229 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:27:23.591229 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:27:23.591229 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:27:23.591229 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:27:23.591229 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:27:23.591229 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application with Lock( Jul 25 05:27:23.591229 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:27:23.591229 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:27:23.591229 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:27:23.591229 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:27:23.592286 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:27:23.592286 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:27:23.592286 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:27:23.592286 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:27:23.592286 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 05:27:23.592286 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 05:27:23.592286 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 05:27:23.592286 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 05:27:23.592286 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 05:27:23.592286 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 05:27:23.592286 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 05:27:23.592286 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Jul 25 05:27:23.592286 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 624/1263] 10.222.0.33 () {56 vars in 1096 bytes} [Mon Jul 25 05:27:23 2022] GET /identity/v3/roles/admin => generated 83 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 25 05:27:23.597666 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ce99b1-dd31-4b8d-80db-2121e7af9a1b None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:27:23.618854 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:27:23.701087 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ce99b1-dd31-4b8d-80db-2121e7af9a1b None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:27:23.734226 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d63c5d-5f81-4fa4-b101-12cd0821d352 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:27:23.734889 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d63c5d-5f81-4fa4-b101-12cd0821d352 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:27:23.735175 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d63c5d-5f81-4fa4-b101-12cd0821d352 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:27:23.735452 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d63c5d-5f81-4fa4-b101-12cd0821d352 None admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:27:23.736202 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1d63c5d-5f81-4fa4-b101-12cd0821d352 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:27:23.736511 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1d63c5d-5f81-4fa4-b101-12cd0821d352 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:27:23.737906 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1d63c5d-5f81-4fa4-b101-12cd0821d352 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:27:23.745780 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 640/1264] 10.222.0.33 () {56 vars in 1099 bytes} [Mon Jul 25 05:27:23 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:27:23.751650 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8475d43-5054-4f21-a94b-a33fff2ce084 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:27:23.817616 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:27:23.851257 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8475d43-5054-4f21-a94b-a33fff2ce084 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:27:23.882694 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f2d0e1c-fc81-4d23-928c-ca8f12f78f03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:27:23.883468 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2d0e1c-fc81-4d23-928c-ca8f12f78f03 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:27:23.883747 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2d0e1c-fc81-4d23-928c-ca8f12f78f03 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:27:23.884015 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2d0e1c-fc81-4d23-928c-ca8f12f78f03 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:27:23.894778 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f2d0e1c-fc81-4d23-928c-ca8f12f78f03 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:27:23.896202 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f2d0e1c-fc81-4d23-928c-ca8f12f78f03 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:27:23.913082 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application [None req-0f2d0e1c-fc81-4d23-928c-ca8f12f78f03 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 05:27:23.913082 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:27:23.913082 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:27:23.913082 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:27:23.913082 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:27:23.913082 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:27:23.913082 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:27:23.913082 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:27:23.913082 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:27:23.913082 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:27:23.913082 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:27:23.913082 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:27:23.913082 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 05:27:23.913082 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 05:27:23.913082 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 05:27:23.914136 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 05:27:23.914136 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:27:23.914136 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:27:23.914136 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:27:23.914136 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:27:23.914136 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:27:23.914136 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:27:23.914136 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:27:23.914136 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application with Lock( Jul 25 05:27:23.914136 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:27:23.914136 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:27:23.914136 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:27:23.914136 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:27:23.914136 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:27:23.914136 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:27:23.914136 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:27:23.914136 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:27:23.915291 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 05:27:23.915291 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 05:27:23.915291 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 05:27:23.915291 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Jul 25 05:27:23.915291 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 625/1265] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Jul 25 05:27:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:27:23.920156 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d63c5d-5f81-4fa4-b101-12cd0821d352 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:27:23.999212 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:27:24.018178 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d63c5d-5f81-4fa4-b101-12cd0821d352 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:27:24.049743 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7454bbd9-5afc-4c75-9d31-8b68d241bbe6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:27:24.050392 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7454bbd9-5afc-4c75-9d31-8b68d241bbe6 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:27:24.050676 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7454bbd9-5afc-4c75-9d31-8b68d241bbe6 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:27:24.050951 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7454bbd9-5afc-4c75-9d31-8b68d241bbe6 None admin] PATH_INFO: `/v3/domains` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:27:24.051670 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7454bbd9-5afc-4c75-9d31-8b68d241bbe6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:27:24.051981 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7454bbd9-5afc-4c75-9d31-8b68d241bbe6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:27:24.053363 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7454bbd9-5afc-4c75-9d31-8b68d241bbe6 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:27:24.065180 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 641/1266] 10.222.0.33 () {56 vars in 1109 bytes} [Mon Jul 25 05:27:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:27:24.071055 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f2d0e1c-fc81-4d23-928c-ca8f12f78f03 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:27:24.170680 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f2d0e1c-fc81-4d23-928c-ca8f12f78f03 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:27:24.190071 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:27:24.205038 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9080ec0b-ac35-4a86-9816-9ab6409a9f20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:27:24.205823 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9080ec0b-ac35-4a86-9816-9ab6409a9f20 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:27:24.206111 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9080ec0b-ac35-4a86-9816-9ab6409a9f20 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:27:24.206391 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9080ec0b-ac35-4a86-9816-9ab6409a9f20 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:27:24.213675 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9080ec0b-ac35-4a86-9816-9ab6409a9f20 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:27:24.215635 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9080ec0b-ac35-4a86-9816-9ab6409a9f20 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:27:24.230893 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application [None req-9080ec0b-ac35-4a86-9816-9ab6409a9f20 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 25 05:27:24.230893 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:27:24.230893 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:27:24.230893 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:27:24.230893 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:27:24.230893 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:27:24.230893 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:27:24.230893 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:27:24.230893 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:27:24.230893 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:27:24.230893 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:27:24.230893 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:27:24.230893 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 25 05:27:24.230893 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 25 05:27:24.230893 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 25 05:27:24.230893 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 25 05:27:24.231995 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:27:24.231995 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:27:24.231995 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 25 05:27:24.231995 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 05:27:24.231995 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 25 05:27:24.231995 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 05:27:24.231995 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:27:24.231995 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:27:24.231995 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:27:24.231995 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:27:24.231995 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:27:24.231995 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application with Lock( Jul 25 05:27:24.231995 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:27:24.231995 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:27:24.231995 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:27:24.231995 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:27:24.231995 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:27:24.231995 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:27:24.233296 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:27:24.233296 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:27:24.233296 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 25 05:27:24.233296 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 25 05:27:24.233296 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 25 05:27:24.233296 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 25 05:27:24.233296 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 25 05:27:24.233296 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 25 05:27:24.233296 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 25 05:27:24.233296 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Jul 25 05:27:24.233296 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 626/1267] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Jul 25 05:27:24 2022] GET /identity/v3/users/placement => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:27:24.237766 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7454bbd9-5afc-4c75-9d31-8b68d241bbe6 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:27:24.335037 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7454bbd9-5afc-4c75-9d31-8b68d241bbe6 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:27:24.368534 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a32c87c7-d09c-4cad-81a6-772403098f52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:27:24.369216 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32c87c7-d09c-4cad-81a6-772403098f52 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:27:24.369502 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32c87c7-d09c-4cad-81a6-772403098f52 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:27:24.369778 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32c87c7-d09c-4cad-81a6-772403098f52 None admin] PATH_INFO: `/v3/users` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:27:24.370608 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a32c87c7-d09c-4cad-81a6-772403098f52 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:27:24.372008 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a32c87c7-d09c-4cad-81a6-772403098f52 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:27:24.373467 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a32c87c7-d09c-4cad-81a6-772403098f52 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:27:24.381703 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:27:24.390285 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 642/1268] 10.222.0.33 () {56 vars in 1143 bytes} [Mon Jul 25 05:27:24 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:27:24.395752 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9080ec0b-ac35-4a86-9816-9ab6409a9f20 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:27:24.491835 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9080ec0b-ac35-4a86-9816-9ab6409a9f20 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:27:24.523695 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a02f1d-2652-4853-beaf-dde1ad0201e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:27:24.524468 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a02f1d-2652-4853-beaf-dde1ad0201e3 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:27:24.524748 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a02f1d-2652-4853-beaf-dde1ad0201e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:27:24.525055 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a02f1d-2652-4853-beaf-dde1ad0201e3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:27:24.536069 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2a02f1d-2652-4853-beaf-dde1ad0201e3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:27:24.537526 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2a02f1d-2652-4853-beaf-dde1ad0201e3 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:27:24.554357 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application [None req-b2a02f1d-2652-4853-beaf-dde1ad0201e3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 05:27:24.554357 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:27:24.554357 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:27:24.554357 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:27:24.554357 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:27:24.554357 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:27:24.554357 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:27:24.554357 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:27:24.554357 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:27:24.554357 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:27:24.554357 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:27:24.554357 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:27:24.554357 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 05:27:24.554357 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 05:27:24.554357 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 05:27:24.555430 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 05:27:24.555430 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:27:24.555430 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:27:24.555430 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:27:24.555430 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:27:24.555430 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:27:24.555430 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:27:24.555430 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:27:24.555430 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application with Lock( Jul 25 05:27:24.555430 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:27:24.555430 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:27:24.555430 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:27:24.555430 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:27:24.555430 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:27:24.555430 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:27:24.555430 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:27:24.555430 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:27:24.556804 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 05:27:24.556804 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 05:27:24.556804 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 05:27:24.556804 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Jul 25 05:27:24.556804 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 627/1269] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Jul 25 05:27:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:27:24.562293 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a32c87c7-d09c-4cad-81a6-772403098f52 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:27:24.670652 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a32c87c7-d09c-4cad-81a6-772403098f52 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:27:24.703201 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dceaae9b-f0c1-4648-b931-fff8551bf4ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:27:24.703850 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dceaae9b-f0c1-4648-b931-fff8551bf4ca None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:27:24.704131 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dceaae9b-f0c1-4648-b931-fff8551bf4ca None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:27:24.704404 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dceaae9b-f0c1-4648-b931-fff8551bf4ca None admin] PATH_INFO: `/v3/domains` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:27:24.705152 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dceaae9b-f0c1-4648-b931-fff8551bf4ca None admin] RBAC: Adding query filter params (name=Default) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:27:24.705454 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dceaae9b-f0c1-4648-b931-fff8551bf4ca None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:27:24.706820 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dceaae9b-f0c1-4648-b931-fff8551bf4ca None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:27:24.718094 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 643/1270] 10.222.0.33 () {56 vars in 1109 bytes} [Mon Jul 25 05:27:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:27:24.723663 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a02f1d-2652-4853-beaf-dde1ad0201e3 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:27:24.754526 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:27:24.822138 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a02f1d-2652-4853-beaf-dde1ad0201e3 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:27:24.854483 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64023ac0-d698-4245-b8d5-2650e4b71323 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:27:24.855893 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64023ac0-d698-4245-b8d5-2650e4b71323 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:27:24.856174 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64023ac0-d698-4245-b8d5-2650e4b71323 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:27:24.856449 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64023ac0-d698-4245-b8d5-2650e4b71323 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:27:24.862925 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64023ac0-d698-4245-b8d5-2650e4b71323 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:27:24.864344 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64023ac0-d698-4245-b8d5-2650e4b71323 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:27:24.877997 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application [None req-64023ac0-d698-4245-b8d5-2650e4b71323 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 05:27:24.877997 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:27:24.877997 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:27:24.877997 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:27:24.877997 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:27:24.877997 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:27:24.877997 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:27:24.877997 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:27:24.877997 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:27:24.877997 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:27:24.877997 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:27:24.877997 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:27:24.877997 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 25 05:27:24.877997 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 25 05:27:24.877997 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 25 05:27:24.879057 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 25 05:27:24.879057 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:27:24.879057 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:27:24.879057 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:27:24.879057 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:27:24.879057 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:27:24.879057 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:27:24.879057 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:27:24.879057 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application with Lock( Jul 25 05:27:24.879057 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:27:24.879057 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:27:24.879057 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:27:24.879057 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:27:24.879057 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:27:24.879057 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:27:24.879057 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:27:24.879057 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:27:24.880140 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 25 05:27:24.880140 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 25 05:27:24.880140 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 25 05:27:24.880140 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 25 05:27:24.880140 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 25 05:27:24.880140 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 25 05:27:24.880140 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 05:27:24.880140 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Jul 25 05:27:24.880140 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 628/1271] 10.222.0.33 () {56 vars in 1111 bytes} [Mon Jul 25 05:27:24 2022] GET /identity/v3/projects/service => generated 88 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:27:24.884902 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dceaae9b-f0c1-4648-b931-fff8551bf4ca None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:27:24.941705 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:27:24.983201 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dceaae9b-f0c1-4648-b931-fff8551bf4ca None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:27:25.014884 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-657fa069-c621-4ffa-90c5-25a89d3fb55b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:27:25.015473 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657fa069-c621-4ffa-90c5-25a89d3fb55b None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:27:25.015756 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657fa069-c621-4ffa-90c5-25a89d3fb55b None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:27:25.016034 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657fa069-c621-4ffa-90c5-25a89d3fb55b None admin] PATH_INFO: `/v3/projects` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:27:25.016837 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-657fa069-c621-4ffa-90c5-25a89d3fb55b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:27:25.017147 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-657fa069-c621-4ffa-90c5-25a89d3fb55b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:27:25.018558 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-657fa069-c621-4ffa-90c5-25a89d3fb55b None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:27:25.030629 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 644/1272] 10.222.0.33 () {56 vars in 1148 bytes} [Mon Jul 25 05:27:24 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:27:25.036514 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64023ac0-d698-4245-b8d5-2650e4b71323 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:27:25.127319 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:27:25.136850 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64023ac0-d698-4245-b8d5-2650e4b71323 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:27:25.170363 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c3b6f24-9177-4137-907b-e082db5b2856 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:27:25.171234 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3b6f24-9177-4137-907b-e082db5b2856 None admin] REQUEST_METHOD: `PUT` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:27:25.171523 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3b6f24-9177-4137-907b-e082db5b2856 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:27:25.171801 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3b6f24-9177-4137-907b-e082db5b2856 None admin] PATH_INFO: `/v3/projects/92afba266cce4d1bad4134c351e74be3/users/317790016e7047fe96af73925de5d872/roles/39e5c666285a42fe8fc2c3afca6260d1` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:27:25.173766 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c3b6f24-9177-4137-907b-e082db5b2856 None admin] RBAC: Authorizing `identity:create_grant(project_id=92afba266cce4d1bad4134c351e74be3, user_id=317790016e7047fe96af73925de5d872, role_id=39e5c666285a42fe8fc2c3afca6260d1)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:27:25.175193 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c3b6f24-9177-4137-907b-e082db5b2856 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:27:25.187309 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 629/1273] 10.222.0.33 () {58 vars in 1439 bytes} [Mon Jul 25 05:27:25 2022] PUT /identity/v3/projects/92afba266cce4d1bad4134c351e74be3/users/317790016e7047fe96af73925de5d872/roles/39e5c666285a42fe8fc2c3afca6260d1 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 05:27:27.171961 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133f204b-1408-47f9-b77e-84ae93dd94d9 None None] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:27:27.172479 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133f204b-1408-47f9-b77e-84ae93dd94d9 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:27:27.172479 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133f204b-1408-47f9-b77e-84ae93dd94d9 None None] PATH_INFO: `/` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:27:27.173835 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 645/1274] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jul 25 05:27:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:27:27.178852 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714671ad-be31-4093-b2c2-4b6c245afe56 None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:27:27.179307 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714671ad-be31-4093-b2c2-4b6c245afe56 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:27:27.179403 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714671ad-be31-4093-b2c2-4b6c245afe56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:27:27.196626 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-714671ad-be31-4093-b2c2-4b6c245afe56 None None] MFA Rules not processed for user `1eae540242d94c10b0ad33960a855f08`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:27:27.197846 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-714671ad-be31-4093-b2c2-4b6c245afe56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:27:27.263538 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 630/1275] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jul 25 05:27:27 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 25 05:27:27.287360 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-133f204b-1408-47f9-b77e-84ae93dd94d9 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:27:27.389065 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-133f204b-1408-47f9-b77e-84ae93dd94d9 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:27:27.406638 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:27:27.423621 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ac30a3-6321-4a1e-89d6-2f6aef8e74a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:27:27.424337 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ac30a3-6321-4a1e-89d6-2f6aef8e74a7 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:27:27.424621 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ac30a3-6321-4a1e-89d6-2f6aef8e74a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:27:27.424925 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ac30a3-6321-4a1e-89d6-2f6aef8e74a7 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:27:27.435174 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55ac30a3-6321-4a1e-89d6-2f6aef8e74a7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 05:27:27.442686 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application [None req-55ac30a3-6321-4a1e-89d6-2f6aef8e74a7 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 05:27:27.442686 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:27:27.442686 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:27:27.442686 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:27:27.442686 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:27:27.442686 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:27:27.442686 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:27:27.442686 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:27:27.442686 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:27:27.442686 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:27:27.442686 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:27:27.442686 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:27:27.442686 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 05:27:27.442686 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 05:27:27.442686 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 05:27:27.442686 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 05:27:27.443827 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 05:27:27.443827 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 05:27:27.443827 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 05:27:27.443827 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 05:27:27.443827 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:27:27.443827 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:27:27.443827 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:27:27.443827 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:27:27.443827 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:27:27.443827 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:27:27.443827 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:27:27.443827 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application with Lock( Jul 25 05:27:27.443827 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:27:27.443827 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:27:27.443827 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:27:27.443827 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:27:27.445074 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:27:27.445074 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:27:27.445074 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:27:27.445074 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:27:27.445074 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 05:27:27.445074 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 05:27:27.445074 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 05:27:27.445074 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 05:27:27.445074 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 05:27:27.445074 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 05:27:27.445074 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 05:27:27.445074 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Jul 25 05:27:27.445074 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 646/1276] 10.222.0.33 () {56 vars in 1096 bytes} [Mon Jul 25 05:27:27 2022] GET /identity/v3/roles/admin => generated 83 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:27:27.450522 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-714671ad-be31-4093-b2c2-4b6c245afe56 None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:27:27.527799 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:27:27.546915 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-714671ad-be31-4093-b2c2-4b6c245afe56 None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:27:27.582948 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5cd4c8f-8281-49bb-9e58-bcfa0864179d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:27:27.583655 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5cd4c8f-8281-49bb-9e58-bcfa0864179d None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:27:27.583982 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5cd4c8f-8281-49bb-9e58-bcfa0864179d None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:27:27.584297 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5cd4c8f-8281-49bb-9e58-bcfa0864179d None admin] PATH_INFO: `/v3/roles` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:27:27.585237 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5cd4c8f-8281-49bb-9e58-bcfa0864179d None admin] RBAC: Adding query filter params (name=admin) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:27:27.585548 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5cd4c8f-8281-49bb-9e58-bcfa0864179d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:27:27.587060 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5cd4c8f-8281-49bb-9e58-bcfa0864179d None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:27:27.596277 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 631/1277] 10.222.0.33 () {56 vars in 1099 bytes} [Mon Jul 25 05:27:27 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:27:27.602132 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ac30a3-6321-4a1e-89d6-2f6aef8e74a7 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:27:27.697489 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ac30a3-6321-4a1e-89d6-2f6aef8e74a7 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:27:27.729660 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b869882-1b44-420f-9071-f6696e48679b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:27:27.730470 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b869882-1b44-420f-9071-f6696e48679b None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:27:27.730751 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b869882-1b44-420f-9071-f6696e48679b None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:27:27.731022 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b869882-1b44-420f-9071-f6696e48679b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:27:27.741129 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b869882-1b44-420f-9071-f6696e48679b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:27:27.742549 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b869882-1b44-420f-9071-f6696e48679b None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:27:27.759553 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application [None req-1b869882-1b44-420f-9071-f6696e48679b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 05:27:27.759553 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:27:27.759553 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:27:27.759553 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:27:27.759553 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:27:27.759553 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:27:27.759553 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:27:27.759553 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:27:27.759553 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:27:27.759553 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:27:27.759553 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:27:27.759553 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:27:27.759553 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 05:27:27.759553 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 05:27:27.759553 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 05:27:27.760579 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 05:27:27.760579 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:27:27.760579 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:27:27.760579 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:27:27.760579 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:27:27.760579 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:27:27.760579 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:27:27.760579 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:27:27.760579 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application with Lock( Jul 25 05:27:27.760579 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:27:27.760579 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:27:27.760579 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:27:27.760579 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:27:27.760579 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:27:27.760579 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:27:27.760579 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:27:27.760579 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:27:27.761702 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 05:27:27.761702 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 05:27:27.761702 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 05:27:27.761702 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Jul 25 05:27:27.761702 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 647/1278] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Jul 25 05:27:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:27:27.767053 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5cd4c8f-8281-49bb-9e58-bcfa0864179d None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:27:27.871524 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5cd4c8f-8281-49bb-9e58-bcfa0864179d None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:27:27.905409 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f06486e8-771e-4858-94c8-a1e878605b5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:27:27.906067 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06486e8-771e-4858-94c8-a1e878605b5c None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:27:27.906353 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06486e8-771e-4858-94c8-a1e878605b5c None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:27:27.906632 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06486e8-771e-4858-94c8-a1e878605b5c None admin] PATH_INFO: `/v3/domains` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:27:27.907354 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f06486e8-771e-4858-94c8-a1e878605b5c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:27:27.907670 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f06486e8-771e-4858-94c8-a1e878605b5c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:27:27.909052 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f06486e8-771e-4858-94c8-a1e878605b5c None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:27:27.921914 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 632/1279] 10.222.0.33 () {56 vars in 1109 bytes} [Mon Jul 25 05:27:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:27:27.927508 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b869882-1b44-420f-9071-f6696e48679b None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:27:27.957463 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:27:28.023918 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b869882-1b44-420f-9071-f6696e48679b None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:27:28.056347 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf3ab08a-cbec-4ff0-b155-d0397dccbe12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:27:28.057149 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3ab08a-cbec-4ff0-b155-d0397dccbe12 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:27:28.057428 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3ab08a-cbec-4ff0-b155-d0397dccbe12 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:27:28.057699 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3ab08a-cbec-4ff0-b155-d0397dccbe12 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:27:28.065688 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf3ab08a-cbec-4ff0-b155-d0397dccbe12 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:27:28.067116 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf3ab08a-cbec-4ff0-b155-d0397dccbe12 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:27:28.081720 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application [None req-bf3ab08a-cbec-4ff0-b155-d0397dccbe12 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 25 05:27:28.081720 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:27:28.081720 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:27:28.081720 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:27:28.081720 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:27:28.081720 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:27:28.081720 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:27:28.081720 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:27:28.081720 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:27:28.081720 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:27:28.081720 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:27:28.081720 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:27:28.081720 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 25 05:27:28.081720 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 25 05:27:28.081720 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 25 05:27:28.081720 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 25 05:27:28.082838 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:27:28.082838 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:27:28.082838 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 25 05:27:28.082838 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 05:27:28.082838 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 25 05:27:28.082838 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 05:27:28.082838 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:27:28.082838 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:27:28.082838 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:27:28.082838 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:27:28.082838 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:27:28.082838 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application with Lock( Jul 25 05:27:28.082838 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:27:28.082838 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:27:28.082838 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:27:28.082838 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:27:28.082838 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:27:28.082838 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:27:28.084000 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:27:28.084000 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:27:28.084000 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 25 05:27:28.084000 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 25 05:27:28.084000 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 25 05:27:28.084000 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 25 05:27:28.084000 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 25 05:27:28.084000 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 25 05:27:28.084000 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 25 05:27:28.084000 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Jul 25 05:27:28.084000 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 648/1280] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Jul 25 05:27:27 2022] GET /identity/v3/users/placement => generated 87 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:27:28.088635 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f06486e8-771e-4858-94c8-a1e878605b5c None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:27:28.093129 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:27:28.184720 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f06486e8-771e-4858-94c8-a1e878605b5c None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:27:28.218489 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc93a0c-34ff-46c9-a473-b201e1120c41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:27:28.219087 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc93a0c-34ff-46c9-a473-b201e1120c41 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:27:28.219375 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc93a0c-34ff-46c9-a473-b201e1120c41 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:27:28.219651 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc93a0c-34ff-46c9-a473-b201e1120c41 None admin] PATH_INFO: `/v3/users` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:27:28.220484 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbc93a0c-34ff-46c9-a473-b201e1120c41 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:27:28.220814 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbc93a0c-34ff-46c9-a473-b201e1120c41 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:27:28.222252 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbc93a0c-34ff-46c9-a473-b201e1120c41 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:27:28.238816 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 633/1281] 10.222.0.33 () {56 vars in 1143 bytes} [Mon Jul 25 05:27:28 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:27:28.244566 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf3ab08a-cbec-4ff0-b155-d0397dccbe12 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:27:28.336308 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:27:28.346349 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf3ab08a-cbec-4ff0-b155-d0397dccbe12 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:27:28.378120 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdce4777-05dd-4d8a-a28c-7f44e0d16dae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:27:28.378758 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdce4777-05dd-4d8a-a28c-7f44e0d16dae None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:27:28.379045 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdce4777-05dd-4d8a-a28c-7f44e0d16dae None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:27:28.379325 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdce4777-05dd-4d8a-a28c-7f44e0d16dae None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:27:28.443919 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdce4777-05dd-4d8a-a28c-7f44e0d16dae None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:27:28.445632 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdce4777-05dd-4d8a-a28c-7f44e0d16dae None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:27:28.530154 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:27:28.540090 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 649/1282] 10.222.0.33 () {58 vars in 1282 bytes} [Mon Jul 25 05:27:28 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 296 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 25 05:27:28.546775 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc93a0c-34ff-46c9-a473-b201e1120c41 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:27:28.614528 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:27:28.652721 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc93a0c-34ff-46c9-a473-b201e1120c41 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:27:28.686356 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d3d5fc-f5de-42df-bc1b-5a74f022ef94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:27:28.687145 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d3d5fc-f5de-42df-bc1b-5a74f022ef94 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:27:28.687432 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d3d5fc-f5de-42df-bc1b-5a74f022ef94 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:27:28.687709 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d3d5fc-f5de-42df-bc1b-5a74f022ef94 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:27:28.697677 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7d3d5fc-f5de-42df-bc1b-5a74f022ef94 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:27:28.699277 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7d3d5fc-f5de-42df-bc1b-5a74f022ef94 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:27:28.717028 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application [None req-b7d3d5fc-f5de-42df-bc1b-5a74f022ef94 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 05:27:28.717028 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:27:28.717028 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:27:28.717028 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:27:28.717028 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:27:28.717028 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:27:28.717028 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:27:28.717028 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:27:28.717028 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:27:28.717028 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:27:28.717028 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:27:28.717028 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:27:28.717028 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 05:27:28.717028 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 05:27:28.717028 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 05:27:28.718100 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 05:27:28.718100 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:27:28.718100 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:27:28.718100 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:27:28.718100 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:27:28.718100 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:27:28.718100 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:27:28.718100 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:27:28.718100 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application with Lock( Jul 25 05:27:28.718100 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:27:28.718100 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:27:28.718100 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:27:28.718100 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:27:28.718100 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:27:28.718100 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:27:28.718100 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:27:28.718100 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:27:28.719093 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 05:27:28.719093 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 05:27:28.719093 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 05:27:28.719093 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Jul 25 05:27:28.719093 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 634/1283] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Jul 25 05:27:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:27:28.724803 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdce4777-05dd-4d8a-a28c-7f44e0d16dae None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:27:28.821137 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdce4777-05dd-4d8a-a28c-7f44e0d16dae None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:27:28.853242 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f233ce2-0272-4c11-b254-e72f9bf19ddf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:27:28.853889 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f233ce2-0272-4c11-b254-e72f9bf19ddf None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:27:28.854172 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f233ce2-0272-4c11-b254-e72f9bf19ddf None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:27:28.854449 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f233ce2-0272-4c11-b254-e72f9bf19ddf None admin] PATH_INFO: `/v3/domains` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:27:28.855670 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f233ce2-0272-4c11-b254-e72f9bf19ddf None admin] RBAC: Adding query filter params (name=Default) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:27:28.856158 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f233ce2-0272-4c11-b254-e72f9bf19ddf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:27:28.857353 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f233ce2-0272-4c11-b254-e72f9bf19ddf None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:27:28.868085 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 650/1284] 10.222.0.33 () {56 vars in 1109 bytes} [Mon Jul 25 05:27:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:27:28.874352 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d3d5fc-f5de-42df-bc1b-5a74f022ef94 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:27:28.971854 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d3d5fc-f5de-42df-bc1b-5a74f022ef94 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:27:28.989565 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:27:29.011470 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ac0266-d7de-4422-a666-888ddeee765d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:27:29.012716 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ac0266-d7de-4422-a666-888ddeee765d None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:27:29.013036 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ac0266-d7de-4422-a666-888ddeee765d None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:27:29.013319 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ac0266-d7de-4422-a666-888ddeee765d None admin] PATH_INFO: `/v3/projects/service` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:27:29.019858 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08ac0266-d7de-4422-a666-888ddeee765d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:27:29.021332 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08ac0266-d7de-4422-a666-888ddeee765d None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:27:29.035040 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application [None req-08ac0266-d7de-4422-a666-888ddeee765d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 05:27:29.035040 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:27:29.035040 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:27:29.035040 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:27:29.035040 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:27:29.035040 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:27:29.035040 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:27:29.035040 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:27:29.035040 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:27:29.035040 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:27:29.035040 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:27:29.035040 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:27:29.035040 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 25 05:27:29.035040 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 25 05:27:29.035040 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 25 05:27:29.036101 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 25 05:27:29.036101 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:27:29.036101 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:27:29.036101 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:27:29.036101 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:27:29.036101 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:27:29.036101 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:27:29.036101 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:27:29.036101 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application with Lock( Jul 25 05:27:29.036101 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:27:29.036101 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:27:29.036101 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:27:29.036101 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:27:29.036101 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:27:29.036101 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:27:29.036101 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:27:29.036101 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:27:29.037432 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 25 05:27:29.037432 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 25 05:27:29.037432 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 25 05:27:29.037432 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 25 05:27:29.037432 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 25 05:27:29.037432 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 25 05:27:29.037432 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 05:27:29.037432 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Jul 25 05:27:29.037432 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 635/1285] 10.222.0.33 () {56 vars in 1111 bytes} [Mon Jul 25 05:27:28 2022] GET /identity/v3/projects/service => generated 88 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:27:29.042054 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f233ce2-0272-4c11-b254-e72f9bf19ddf None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:27:29.091158 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:27:29.140487 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f233ce2-0272-4c11-b254-e72f9bf19ddf None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:27:29.173316 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa5ffb6-4fcf-4097-99ae-596af89b2d07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:27:29.173966 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa5ffb6-4fcf-4097-99ae-596af89b2d07 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:27:29.174251 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa5ffb6-4fcf-4097-99ae-596af89b2d07 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:27:29.174529 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa5ffb6-4fcf-4097-99ae-596af89b2d07 None admin] PATH_INFO: `/v3/projects` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:27:29.175314 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fa5ffb6-4fcf-4097-99ae-596af89b2d07 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:27:29.175626 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fa5ffb6-4fcf-4097-99ae-596af89b2d07 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:27:29.177099 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fa5ffb6-4fcf-4097-99ae-596af89b2d07 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:27:29.189330 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 651/1286] 10.222.0.33 () {56 vars in 1148 bytes} [Mon Jul 25 05:27:29 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:27:29.195629 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ac0266-d7de-4422-a666-888ddeee765d None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:27:29.291075 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ac0266-d7de-4422-a666-888ddeee765d None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:27:29.323216 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ac1e2e-aedb-42fe-8858-415fcdf3326e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:27:29.323860 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ac1e2e-aedb-42fe-8858-415fcdf3326e None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:27:29.324299 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ac1e2e-aedb-42fe-8858-415fcdf3326e None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:27:29.324653 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ac1e2e-aedb-42fe-8858-415fcdf3326e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:27:29.325512 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36ac1e2e-aedb-42fe-8858-415fcdf3326e None admin] RBAC: Adding query filter params (role.id=39e5c666285a42fe8fc2c3afca6260d1, scope.project.id=92afba266cce4d1bad4134c351e74be3, user.id=317790016e7047fe96af73925de5d872) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:27:29.325827 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36ac1e2e-aedb-42fe-8858-415fcdf3326e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:27:29.327239 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36ac1e2e-aedb-42fe-8858-415fcdf3326e None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:27:29.332836 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 636/1287] 10.222.0.33 () {56 vars in 1374 bytes} [Mon Jul 25 05:27:29 2022] GET /identity/v3/role_assignments?user.id=317790016e7047fe96af73925de5d872&scope.project.id=92afba266cce4d1bad4134c351e74be3&role.id=39e5c666285a42fe8fc2c3afca6260d1 => generated 608 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:27:31.381486 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a127bee0-71ea-41b8-87b8-ea94489aad15 None None] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:27:31.382116 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a127bee0-71ea-41b8-87b8-ea94489aad15 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:27:31.382116 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a127bee0-71ea-41b8-87b8-ea94489aad15 None None] PATH_INFO: `/` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:27:31.383433 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 652/1288] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jul 25 05:27:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:27:31.389268 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41441c5-e7ec-42df-ac20-002cf45f5b8b None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:27:31.389675 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41441c5-e7ec-42df-ac20-002cf45f5b8b None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:27:31.389854 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41441c5-e7ec-42df-ac20-002cf45f5b8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:27:31.406830 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-d41441c5-e7ec-42df-ac20-002cf45f5b8b None None] MFA Rules not processed for user `1eae540242d94c10b0ad33960a855f08`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:27:31.408004 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-d41441c5-e7ec-42df-ac20-002cf45f5b8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:27:31.437436 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:27:31.474993 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 637/1289] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jul 25 05:27:31 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 25 05:27:31.497737 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a127bee0-71ea-41b8-87b8-ea94489aad15 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:27:31.601543 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a127bee0-71ea-41b8-87b8-ea94489aad15 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:27:31.614559 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:27:31.641328 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a87dc5-0b0e-41e6-8b06-d49705992dcd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:27:31.642059 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a87dc5-0b0e-41e6-8b06-d49705992dcd None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:27:31.642343 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a87dc5-0b0e-41e6-8b06-d49705992dcd None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:27:31.642620 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a87dc5-0b0e-41e6-8b06-d49705992dcd None admin] PATH_INFO: `/v3/services/placement` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:27:31.643555 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3a87dc5-0b0e-41e6-8b06-d49705992dcd None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:27:31.645165 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3a87dc5-0b0e-41e6-8b06-d49705992dcd None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:27:31.660716 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application [None req-f3a87dc5-0b0e-41e6-8b06-d49705992dcd None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jul 25 05:27:31.660716 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:27:31.660716 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 25 05:27:31.660716 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 25 05:27:31.660716 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 25 05:27:31.660716 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 25 05:27:31.660716 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 25 05:27:31.660716 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 25 05:27:31.660716 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 25 05:27:31.660716 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Jul 25 05:27:31.660716 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 25 05:27:31.660716 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Jul 25 05:27:31.660716 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:27:31.660716 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:27:31.660716 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:27:31.660716 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:27:31.660716 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:27:31.661875 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:27:31.661875 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:27:31.661875 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:27:31.661875 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:27:31.661875 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:27:31.661875 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:27:31.661875 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 25 05:27:31.661875 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 25 05:27:31.661875 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 25 05:27:31.661875 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 25 05:27:31.661875 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:27:31.661875 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:27:31.661875 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:27:31.661875 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:27:31.661875 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:27:31.661875 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:27:31.662822 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:27:31.662822 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application with Lock( Jul 25 05:27:31.662822 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:27:31.662822 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:27:31.662822 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:27:31.662822 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:27:31.662822 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:27:31.662822 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:27:31.662822 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:27:31.662822 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:27:31.662822 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 25 05:27:31.662822 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 25 05:27:31.662822 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 25 05:27:31.662822 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Jul 25 05:27:31.662822 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 653/1290] 10.222.0.33 () {56 vars in 1117 bytes} [Mon Jul 25 05:27:31 2022] GET /identity/v3/services/placement => generated 90 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:27:31.669114 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d41441c5-e7ec-42df-ac20-002cf45f5b8b None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:27:31.765586 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d41441c5-e7ec-42df-ac20-002cf45f5b8b None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:27:31.798503 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3dd5f4d-f85c-4fa0-b61a-7e6e1d531c17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:27:31.799169 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3dd5f4d-f85c-4fa0-b61a-7e6e1d531c17 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:27:31.799456 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3dd5f4d-f85c-4fa0-b61a-7e6e1d531c17 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:27:31.799734 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3dd5f4d-f85c-4fa0-b61a-7e6e1d531c17 None admin] PATH_INFO: `/v3/services` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:27:31.800449 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3dd5f4d-f85c-4fa0-b61a-7e6e1d531c17 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:27:31.800771 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3dd5f4d-f85c-4fa0-b61a-7e6e1d531c17 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:27:31.802175 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3dd5f4d-f85c-4fa0-b61a-7e6e1d531c17 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:27:31.807193 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 638/1291] 10.222.0.33 () {56 vars in 1116 bytes} [Mon Jul 25 05:27:31 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:27:31.813573 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a87dc5-0b0e-41e6-8b06-d49705992dcd None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:27:31.909419 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a87dc5-0b0e-41e6-8b06-d49705992dcd None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:27:31.942426 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d368d64-ffab-4766-a6b1-9ea5dc821711 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:27:31.943066 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d368d64-ffab-4766-a6b1-9ea5dc821711 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:27:31.943348 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d368d64-ffab-4766-a6b1-9ea5dc821711 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:27:31.943625 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d368d64-ffab-4766-a6b1-9ea5dc821711 None admin] PATH_INFO: `/v3/services` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:27:31.944320 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d368d64-ffab-4766-a6b1-9ea5dc821711 None admin] RBAC: Adding query filter params (type=placement) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:27:31.944623 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d368d64-ffab-4766-a6b1-9ea5dc821711 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:27:31.946310 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d368d64-ffab-4766-a6b1-9ea5dc821711 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:27:31.950660 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 654/1292] 10.222.0.33 () {56 vars in 1116 bytes} [Mon Jul 25 05:27:31 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:27:33.953757 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57af4ca3-4dc2-4521-9e5c-1fd431e811c9 None None] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:27:33.954334 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57af4ca3-4dc2-4521-9e5c-1fd431e811c9 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:27:33.954334 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57af4ca3-4dc2-4521-9e5c-1fd431e811c9 None None] PATH_INFO: `/` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:27:33.955699 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 639/1293] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jul 25 05:27:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:27:33.960918 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41b63b4-9153-4e06-a3a4-91248de4da5a None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:27:33.961307 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41b63b4-9153-4e06-a3a4-91248de4da5a None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:27:33.961485 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41b63b4-9153-4e06-a3a4-91248de4da5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:27:33.978418 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-e41b63b4-9153-4e06-a3a4-91248de4da5a None None] MFA Rules not processed for user `1eae540242d94c10b0ad33960a855f08`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:27:33.979605 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-e41b63b4-9153-4e06-a3a4-91248de4da5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:27:34.008391 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:27:34.045060 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 655/1294] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jul 25 05:27:33 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 25 05:27:34.067517 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57af4ca3-4dc2-4521-9e5c-1fd431e811c9 None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:27:34.099155 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:27:34.167675 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57af4ca3-4dc2-4521-9e5c-1fd431e811c9 None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:27:34.200880 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c36e0c2-cb15-4f8d-b143-8e578ec2d03f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:27:34.201533 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c36e0c2-cb15-4f8d-b143-8e578ec2d03f None admin] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:27:34.201822 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c36e0c2-cb15-4f8d-b143-8e578ec2d03f None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:27:34.202101 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c36e0c2-cb15-4f8d-b143-8e578ec2d03f None admin] PATH_INFO: `/v3/services` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:27:34.202836 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c36e0c2-cb15-4f8d-b143-8e578ec2d03f None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:27:34.204203 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c36e0c2-cb15-4f8d-b143-8e578ec2d03f None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:27:34.216447 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 640/1295] 10.222.0.33 () {60 vars in 1141 bytes} [Mon Jul 25 05:27:34 2022] POST /identity/v3/services => generated 246 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 05:27:36.238913 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8716b3b0-12bc-439d-a07f-211ce1605d38 None None] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:27:36.239430 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8716b3b0-12bc-439d-a07f-211ce1605d38 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:27:36.239430 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8716b3b0-12bc-439d-a07f-211ce1605d38 None None] PATH_INFO: `/` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:27:36.240767 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 656/1296] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jul 25 05:27:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:27:36.246151 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841739cd-dbf4-4395-8515-2958573e2160 None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:27:36.246599 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841739cd-dbf4-4395-8515-2958573e2160 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:27:36.246851 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841739cd-dbf4-4395-8515-2958573e2160 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:27:36.263732 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-841739cd-dbf4-4395-8515-2958573e2160 None None] MFA Rules not processed for user `1eae540242d94c10b0ad33960a855f08`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:27:36.264977 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-841739cd-dbf4-4395-8515-2958573e2160 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:27:36.336788 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 641/1297] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jul 25 05:27:36 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 25 05:27:36.360173 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8716b3b0-12bc-439d-a07f-211ce1605d38 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:27:36.457203 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8716b3b0-12bc-439d-a07f-211ce1605d38 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:27:36.489575 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcbb0e64-9a70-4b31-9452-bb3976e617ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:27:36.490304 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcbb0e64-9a70-4b31-9452-bb3976e617ec None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:27:36.490588 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcbb0e64-9a70-4b31-9452-bb3976e617ec None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:27:36.490860 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcbb0e64-9a70-4b31-9452-bb3976e617ec None admin] PATH_INFO: `/v3/services/placement` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:27:36.491494 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcbb0e64-9a70-4b31-9452-bb3976e617ec None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:27:36.492869 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcbb0e64-9a70-4b31-9452-bb3976e617ec None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:27:36.504415 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application [None req-fcbb0e64-9a70-4b31-9452-bb3976e617ec None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jul 25 05:27:36.504415 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:27:36.504415 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 25 05:27:36.504415 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 25 05:27:36.504415 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 25 05:27:36.504415 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 25 05:27:36.504415 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 25 05:27:36.504415 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 25 05:27:36.504415 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 25 05:27:36.504415 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Jul 25 05:27:36.504415 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 25 05:27:36.504415 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Jul 25 05:27:36.504415 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:27:36.504415 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:27:36.504415 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:27:36.504415 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:27:36.504415 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:27:36.505623 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:27:36.505623 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:27:36.505623 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:27:36.505623 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:27:36.505623 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:27:36.505623 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:27:36.505623 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 25 05:27:36.505623 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 25 05:27:36.505623 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 25 05:27:36.505623 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 25 05:27:36.505623 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:27:36.505623 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:27:36.505623 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:27:36.505623 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:27:36.505623 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:27:36.505623 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:27:36.506464 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:27:36.506464 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application with Lock( Jul 25 05:27:36.506464 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:27:36.506464 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:27:36.506464 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:27:36.506464 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:27:36.506464 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:27:36.506464 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:27:36.506464 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:27:36.506464 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:27:36.506464 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 25 05:27:36.506464 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 25 05:27:36.506464 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 25 05:27:36.506464 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Jul 25 05:27:36.506464 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 657/1298] 10.222.0.33 () {56 vars in 1117 bytes} [Mon Jul 25 05:27:36 2022] GET /identity/v3/services/placement => generated 90 bytes in 146 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:27:36.511533 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-841739cd-dbf4-4395-8515-2958573e2160 None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:27:36.530745 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:27:36.615028 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-841739cd-dbf4-4395-8515-2958573e2160 None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:27:36.649382 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf13e38b-71ac-4e2c-a643-f5c92a5ff357 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:27:36.650051 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf13e38b-71ac-4e2c-a643-f5c92a5ff357 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:27:36.650336 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf13e38b-71ac-4e2c-a643-f5c92a5ff357 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:27:36.650613 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf13e38b-71ac-4e2c-a643-f5c92a5ff357 None admin] PATH_INFO: `/v3/services` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:27:36.651320 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf13e38b-71ac-4e2c-a643-f5c92a5ff357 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:27:36.651629 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf13e38b-71ac-4e2c-a643-f5c92a5ff357 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:27:36.653985 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf13e38b-71ac-4e2c-a643-f5c92a5ff357 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:27:36.659059 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 642/1299] 10.222.0.33 () {56 vars in 1116 bytes} [Mon Jul 25 05:27:36 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:27:36.665036 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcbb0e64-9a70-4b31-9452-bb3976e617ec None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:27:36.701049 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:27:36.766606 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcbb0e64-9a70-4b31-9452-bb3976e617ec None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:27:36.798467 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c446e4cf-3115-4a1b-aa30-9877f03a4c44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:27:36.799128 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c446e4cf-3115-4a1b-aa30-9877f03a4c44 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:27:36.799418 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c446e4cf-3115-4a1b-aa30-9877f03a4c44 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:27:36.799700 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c446e4cf-3115-4a1b-aa30-9877f03a4c44 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:27:36.800498 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c446e4cf-3115-4a1b-aa30-9877f03a4c44 None admin] RBAC: Adding query filter params (interface=public, service_id=770534c8b2494910aa27b933f5aaadf3, region_id=RegionOne) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:27:36.800860 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c446e4cf-3115-4a1b-aa30-9877f03a4c44 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:27:36.802207 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c446e4cf-3115-4a1b-aa30-9877f03a4c44 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:27:36.807470 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 658/1300] 10.222.0.33 () {56 vars in 1251 bytes} [Mon Jul 25 05:27:36 2022] GET /identity/v3/endpoints?service_id=770534c8b2494910aa27b933f5aaadf3&interface=public®ion_id=RegionOne => generated 194 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:27:36.813666 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf13e38b-71ac-4e2c-a643-f5c92a5ff357 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:27:36.897893 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:27:36.915410 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf13e38b-71ac-4e2c-a643-f5c92a5ff357 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:27:36.947873 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e2ed46-4ed0-40bf-973f-204088a65885 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:27:36.948518 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e2ed46-4ed0-40bf-973f-204088a65885 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:27:36.948817 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e2ed46-4ed0-40bf-973f-204088a65885 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:27:36.949101 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e2ed46-4ed0-40bf-973f-204088a65885 None admin] PATH_INFO: `/v3/services` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:27:36.950426 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8e2ed46-4ed0-40bf-973f-204088a65885 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:27:36.951788 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8e2ed46-4ed0-40bf-973f-204088a65885 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:27:36.958550 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 643/1301] 10.222.0.33 () {56 vars in 1087 bytes} [Mon Jul 25 05:27:36 2022] GET /identity/v3/services => generated 2191 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 05:27:38.976844 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8eec14-205f-4e76-ae57-fe49618b118e None None] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:27:38.977366 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8eec14-205f-4e76-ae57-fe49618b118e None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:27:38.977366 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8eec14-205f-4e76-ae57-fe49618b118e None None] PATH_INFO: `/` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:27:38.978909 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 659/1302] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jul 25 05:27:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:27:38.984580 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e48cbc-da37-4db9-8632-c50c446a0506 None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:27:38.984990 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e48cbc-da37-4db9-8632-c50c446a0506 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:27:38.985170 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e48cbc-da37-4db9-8632-c50c446a0506 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:27:39.005425 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-87e48cbc-da37-4db9-8632-c50c446a0506 None None] MFA Rules not processed for user `1eae540242d94c10b0ad33960a855f08`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:27:39.006618 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-87e48cbc-da37-4db9-8632-c50c446a0506 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:27:39.071874 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 644/1303] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jul 25 05:27:38 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 25 05:27:39.095017 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc8eec14-205f-4e76-ae57-fe49618b118e None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:27:39.188063 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:27:39.194908 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc8eec14-205f-4e76-ae57-fe49618b118e None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:27:39.228074 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9469900-6e68-427c-81f2-3f7cde946ad0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:27:39.228892 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9469900-6e68-427c-81f2-3f7cde946ad0 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:27:39.229182 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9469900-6e68-427c-81f2-3f7cde946ad0 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:27:39.229459 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9469900-6e68-427c-81f2-3f7cde946ad0 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:27:39.230107 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9469900-6e68-427c-81f2-3f7cde946ad0 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:27:39.232029 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9469900-6e68-427c-81f2-3f7cde946ad0 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:27:39.243655 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application [None req-f9469900-6e68-427c-81f2-3f7cde946ad0 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jul 25 05:27:39.243655 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:27:39.243655 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 25 05:27:39.243655 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 25 05:27:39.243655 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 25 05:27:39.243655 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 25 05:27:39.243655 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 25 05:27:39.243655 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 25 05:27:39.243655 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 25 05:27:39.243655 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Jul 25 05:27:39.243655 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 25 05:27:39.243655 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Jul 25 05:27:39.243655 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:27:39.243655 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:27:39.243655 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:27:39.243655 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:27:39.243655 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:27:39.244841 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:27:39.244841 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:27:39.244841 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:27:39.244841 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:27:39.244841 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:27:39.244841 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:27:39.244841 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 25 05:27:39.244841 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 25 05:27:39.244841 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 25 05:27:39.244841 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 25 05:27:39.244841 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:27:39.244841 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:27:39.244841 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:27:39.244841 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:27:39.244841 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:27:39.244841 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:27:39.245953 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:27:39.245953 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application with Lock( Jul 25 05:27:39.245953 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:27:39.245953 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:27:39.245953 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:27:39.245953 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:27:39.245953 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:27:39.245953 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:27:39.245953 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:27:39.245953 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:27:39.245953 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 25 05:27:39.245953 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 25 05:27:39.245953 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 25 05:27:39.245953 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Jul 25 05:27:39.245953 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 660/1304] 10.222.0.33 () {56 vars in 1117 bytes} [Mon Jul 25 05:27:39 2022] GET /identity/v3/services/placement => generated 90 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:27:39.251827 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e48cbc-da37-4db9-8632-c50c446a0506 None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:27:39.326262 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:27:39.348625 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e48cbc-da37-4db9-8632-c50c446a0506 None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:27:39.380652 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0555b73-86d2-47cf-8f07-407844919dad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:27:39.381333 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0555b73-86d2-47cf-8f07-407844919dad None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:27:39.381621 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0555b73-86d2-47cf-8f07-407844919dad None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:27:39.381899 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0555b73-86d2-47cf-8f07-407844919dad None admin] PATH_INFO: `/v3/services` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:27:39.382601 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0555b73-86d2-47cf-8f07-407844919dad None admin] RBAC: Adding query filter params (name=placement) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:27:39.382909 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0555b73-86d2-47cf-8f07-407844919dad None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:27:39.384269 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0555b73-86d2-47cf-8f07-407844919dad None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:27:39.389957 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 645/1305] 10.222.0.33 () {56 vars in 1116 bytes} [Mon Jul 25 05:27:39 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:27:39.395601 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9469900-6e68-427c-81f2-3f7cde946ad0 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:27:39.493468 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9469900-6e68-427c-81f2-3f7cde946ad0 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:27:39.526861 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc93df7d-7ad1-4a4f-b5f3-f6eb14dfe959 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:27:39.528146 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc93df7d-7ad1-4a4f-b5f3-f6eb14dfe959 None admin] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:27:39.528439 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc93df7d-7ad1-4a4f-b5f3-f6eb14dfe959 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:27:39.528711 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc93df7d-7ad1-4a4f-b5f3-f6eb14dfe959 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:27:39.529473 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc93df7d-7ad1-4a4f-b5f3-f6eb14dfe959 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:27:39.530833 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc93df7d-7ad1-4a4f-b5f3-f6eb14dfe959 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:27:39.548141 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 661/1306] 10.222.0.33 () {60 vars in 1144 bytes} [Mon Jul 25 05:27:39 2022] POST /identity/v3/endpoints => generated 331 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 05:27:49.386727 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=30 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:27:49.544285 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:28:42.402727 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9f3089-d088-45d2-a6e8-b37aa60860df None None] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:28:42.404483 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9f3089-d088-45d2-a6e8-b37aa60860df None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:28:42.405295 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9f3089-d088-45d2-a6e8-b37aa60860df None None] PATH_INFO: `/` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:28:42.409126 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 646/1307] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jul 25 05:28:42 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:28:42.416048 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d8c947-44ce-4ef0-bb4a-a81d5b599e2e None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:28:42.417600 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d8c947-44ce-4ef0-bb4a-a81d5b599e2e None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:28:42.418620 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d8c947-44ce-4ef0-bb4a-a81d5b599e2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:28:42.505534 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-97d8c947-44ce-4ef0-bb4a-a81d5b599e2e None None] MFA Rules not processed for user `1eae540242d94c10b0ad33960a855f08`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:28:42.507036 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-97d8c947-44ce-4ef0-bb4a-a81d5b599e2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:28:42.631536 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 662/1308] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jul 25 05:28:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 219 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:28:42.682093 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0a4f2d-6c7d-4bae-bf93-09ec80783459 None None] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:28:42.682385 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0a4f2d-6c7d-4bae-bf93-09ec80783459 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:28:42.682986 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0a4f2d-6c7d-4bae-bf93-09ec80783459 None None] PATH_INFO: `/` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:28:42.684387 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 647/1309] 10.222.0.33 () {54 vars in 965 bytes} [Mon Jul 25 05:28:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:28:42.691615 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4636d2e9-9616-405d-8b21-906a7400bc14 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:28:42.691985 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4636d2e9-9616-405d-8b21-906a7400bc14 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:28:42.692339 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4636d2e9-9616-405d-8b21-906a7400bc14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:28:42.769220 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-4636d2e9-9616-405d-8b21-906a7400bc14 None None] MFA Rules not processed for user `b2931200f99e43d79a50a9f6e02738e6`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:28:42.770448 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-4636d2e9-9616-405d-8b21-906a7400bc14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:28:42.792297 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:28:42.860276 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 663/1310] 10.222.0.33 () {58 vars in 1062 bytes} [Mon Jul 25 05:28:42 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:28:42.869138 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa0a4f2d-6c7d-4bae-bf93-09ec80783459 None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:28:42.870883 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-fa0a4f2d-6c7d-4bae-bf93-09ec80783459 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:28:42.989361 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa0a4f2d-6c7d-4bae-bf93-09ec80783459 None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:28:43.067647 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d515bcd6-4868-4689-b187-33dde3d44abf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b2931200f99e43d79a50a9f6e02738e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92afba266cce4d1bad4134c351e74be3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:28:43.068409 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d515bcd6-4868-4689-b187-33dde3d44abf service neutron] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:28:43.068697 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d515bcd6-4868-4689-b187-33dde3d44abf service neutron] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:28:43.069007 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d515bcd6-4868-4689-b187-33dde3d44abf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:28:43.070656 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-d515bcd6-4868-4689-b187-33dde3d44abf service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:28:43.127736 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:28:43.184973 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d515bcd6-4868-4689-b187-33dde3d44abf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:28:43.186414 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d515bcd6-4868-4689-b187-33dde3d44abf service neutron] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:28:43.206619 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 648/1311] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jul 25 05:28:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 338 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 05:28:45.595649 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bbe6c56-df2c-42b1-bf0f-17af4f879c24 None None] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:28:45.596244 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bbe6c56-df2c-42b1-bf0f-17af4f879c24 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:28:45.596244 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bbe6c56-df2c-42b1-bf0f-17af4f879c24 None None] PATH_INFO: `/` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:28:45.597817 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 664/1312] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jul 25 05:28:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:28:45.602891 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0bc416-96a9-4c1b-a993-fe5c1ef47f3a None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:28:45.603299 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0bc416-96a9-4c1b-a993-fe5c1ef47f3a None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:28:45.603563 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0bc416-96a9-4c1b-a993-fe5c1ef47f3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:28:45.685135 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-7e0bc416-96a9-4c1b-a993-fe5c1ef47f3a None None] MFA Rules not processed for user `1eae540242d94c10b0ad33960a855f08`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:28:45.686351 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-7e0bc416-96a9-4c1b-a993-fe5c1ef47f3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:28:45.784415 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 649/1313] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jul 25 05:28:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 05:28:45.823336 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bbe6c56-df2c-42b1-bf0f-17af4f879c24 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:28:45.824929 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-2bbe6c56-df2c-42b1-bf0f-17af4f879c24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:28:45.939207 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bbe6c56-df2c-42b1-bf0f-17af4f879c24 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:28:46.013844 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:28:46.016491 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab12aee-535f-4833-9c0b-f5dabfe6f904 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b2931200f99e43d79a50a9f6e02738e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92afba266cce4d1bad4134c351e74be3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:28:46.017358 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab12aee-535f-4833-9c0b-f5dabfe6f904 service neutron] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:28:46.017824 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab12aee-535f-4833-9c0b-f5dabfe6f904 service neutron] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:28:46.018293 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab12aee-535f-4833-9c0b-f5dabfe6f904 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:28:46.019954 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-7ab12aee-535f-4833-9c0b-f5dabfe6f904 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:28:46.129508 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ab12aee-535f-4833-9c0b-f5dabfe6f904 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:28:46.130990 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ab12aee-535f-4833-9c0b-f5dabfe6f904 service neutron] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:28:46.151247 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 665/1314] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jul 25 05:28:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 05:28:48.509301 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3a0693-badb-444e-81eb-3050e93c463a None None] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:28:48.509825 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3a0693-badb-444e-81eb-3050e93c463a None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:28:48.509825 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3a0693-badb-444e-81eb-3050e93c463a None None] PATH_INFO: `/` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:28:48.511156 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 650/1315] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jul 25 05:28:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:28:48.516926 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f8b98f-0af7-4b95-9f26-3138e7c5056c None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:28:48.517350 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f8b98f-0af7-4b95-9f26-3138e7c5056c None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:28:48.517482 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f8b98f-0af7-4b95-9f26-3138e7c5056c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:28:48.583893 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-e3f8b98f-0af7-4b95-9f26-3138e7c5056c None None] MFA Rules not processed for user `c25734c1f07644de9355e54cfd5f65cc`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:28:48.585196 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-e3f8b98f-0af7-4b95-9f26-3138e7c5056c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:28:48.641734 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:28:48.686039 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 666/1316] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jul 25 05:28:48 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 05:28:48.726379 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a3a0693-badb-444e-81eb-3050e93c463a None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:28:48.727899 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-5a3a0693-badb-444e-81eb-3050e93c463a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:28:48.737910 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:28:48.834183 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a3a0693-badb-444e-81eb-3050e93c463a None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:28:48.906640 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9816be43-66b1-4cd9-a217-d61415cead1a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b2931200f99e43d79a50a9f6e02738e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92afba266cce4d1bad4134c351e74be3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:28:48.907289 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9816be43-66b1-4cd9-a217-d61415cead1a service neutron] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:28:48.907574 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9816be43-66b1-4cd9-a217-d61415cead1a service neutron] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:28:48.907850 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9816be43-66b1-4cd9-a217-d61415cead1a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:28:48.909521 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-9816be43-66b1-4cd9-a217-d61415cead1a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:28:48.971075 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:28:49.017100 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9816be43-66b1-4cd9-a217-d61415cead1a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:28:49.018541 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9816be43-66b1-4cd9-a217-d61415cead1a service neutron] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:28:49.036384 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 651/1317] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jul 25 05:28:48 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 05:28:51.600449 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae4e0fe-11a7-42f0-a14e-9b4cb5c42dee None None] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:28:51.601028 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae4e0fe-11a7-42f0-a14e-9b4cb5c42dee None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:28:51.601028 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae4e0fe-11a7-42f0-a14e-9b4cb5c42dee None None] PATH_INFO: `/` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:28:51.602441 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 667/1318] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jul 25 05:28:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:28:51.607725 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f113bb7-c1e3-4dbc-b954-2a4663dbc8ae None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:28:51.608140 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f113bb7-c1e3-4dbc-b954-2a4663dbc8ae None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:28:51.608309 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f113bb7-c1e3-4dbc-b954-2a4663dbc8ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:28:51.672642 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-2f113bb7-c1e3-4dbc-b954-2a4663dbc8ae None None] MFA Rules not processed for user `1eae540242d94c10b0ad33960a855f08`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:28:51.673874 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-2f113bb7-c1e3-4dbc-b954-2a4663dbc8ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:28:51.767009 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 652/1319] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jul 25 05:28:51 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:28:51.820874 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc937dd1-0f03-4b53-ac43-121908239123 None None] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:28:51.821170 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc937dd1-0f03-4b53-ac43-121908239123 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:28:51.821456 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc937dd1-0f03-4b53-ac43-121908239123 None None] PATH_INFO: `/` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:28:51.822841 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 668/1320] 10.222.0.33 () {54 vars in 965 bytes} [Mon Jul 25 05:28:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:28:51.828928 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa3abc0-f985-4ca9-852b-969d23ef465e None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:28:51.829220 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa3abc0-f985-4ca9-852b-969d23ef465e None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:28:51.829501 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa3abc0-f985-4ca9-852b-969d23ef465e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:28:51.851284 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:28:51.904892 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-6fa3abc0-f985-4ca9-852b-969d23ef465e None None] MFA Rules not processed for user `b2931200f99e43d79a50a9f6e02738e6`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:28:51.906905 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-6fa3abc0-f985-4ca9-852b-969d23ef465e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:28:51.982209 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 653/1321] 10.222.0.33 () {58 vars in 1062 bytes} [Mon Jul 25 05:28:51 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:28:51.991181 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc937dd1-0f03-4b53-ac43-121908239123 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:28:51.992707 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-fc937dd1-0f03-4b53-ac43-121908239123 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:28:52.095110 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc937dd1-0f03-4b53-ac43-121908239123 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:28:52.167821 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b09cac5-0347-4a1d-b5e9-116686523ef6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b2931200f99e43d79a50a9f6e02738e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92afba266cce4d1bad4134c351e74be3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:28:52.168393 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b09cac5-0347-4a1d-b5e9-116686523ef6 service neutron] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:28:52.168684 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b09cac5-0347-4a1d-b5e9-116686523ef6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:28:52.168990 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b09cac5-0347-4a1d-b5e9-116686523ef6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:28:52.170629 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-3b09cac5-0347-4a1d-b5e9-116686523ef6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:28:52.178263 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:28:52.277251 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b09cac5-0347-4a1d-b5e9-116686523ef6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:28:52.278679 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b09cac5-0347-4a1d-b5e9-116686523ef6 service neutron] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:28:52.297172 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 669/1322] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jul 25 05:28:51 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 05:28:55.472389 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca7ed09-dc7a-484d-95aa-a85e0b15aeda None None] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:28:55.472969 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca7ed09-dc7a-484d-95aa-a85e0b15aeda None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:28:55.473089 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca7ed09-dc7a-484d-95aa-a85e0b15aeda None None] PATH_INFO: `/` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:28:55.474476 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 654/1323] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jul 25 05:28:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:28:55.479816 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54274951-83d2-4cad-9da5-9df5972d42d9 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:28:55.480233 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54274951-83d2-4cad-9da5-9df5972d42d9 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:28:55.480379 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54274951-83d2-4cad-9da5-9df5972d42d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:28:55.545835 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-54274951-83d2-4cad-9da5-9df5972d42d9 None None] MFA Rules not processed for user `1eae540242d94c10b0ad33960a855f08`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:28:55.547206 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-54274951-83d2-4cad-9da5-9df5972d42d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:28:55.608613 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:28:55.652146 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 670/1324] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jul 25 05:28:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:28:55.690688 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca7ed09-dc7a-484d-95aa-a85e0b15aeda None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:28:55.692230 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-8ca7ed09-dc7a-484d-95aa-a85e0b15aeda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:28:55.803939 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca7ed09-dc7a-484d-95aa-a85e0b15aeda None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:28:55.817162 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:28:55.879173 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3286e31-f717-40fe-8275-c946fa723f21 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b2931200f99e43d79a50a9f6e02738e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92afba266cce4d1bad4134c351e74be3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:28:55.879813 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3286e31-f717-40fe-8275-c946fa723f21 service neutron] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:28:55.880094 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3286e31-f717-40fe-8275-c946fa723f21 service neutron] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:28:55.880366 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3286e31-f717-40fe-8275-c946fa723f21 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:28:55.882029 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-c3286e31-f717-40fe-8275-c946fa723f21 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:28:55.993533 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3286e31-f717-40fe-8275-c946fa723f21 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:28:55.994967 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3286e31-f717-40fe-8275-c946fa723f21 service neutron] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:28:56.014342 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 655/1325] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jul 25 05:28:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 05:28:58.124059 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4b4d0f-9879-43b7-956e-61b5c63a212e None None] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:28:58.124656 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4b4d0f-9879-43b7-956e-61b5c63a212e None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:28:58.124656 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4b4d0f-9879-43b7-956e-61b5c63a212e None None] PATH_INFO: `/` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:28:58.126062 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 671/1326] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jul 25 05:28:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:28:58.131451 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae941c7a-a064-4558-b911-2358c588ffb7 None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:28:58.131876 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae941c7a-a064-4558-b911-2358c588ffb7 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:28:58.131991 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae941c7a-a064-4558-b911-2358c588ffb7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:28:58.178904 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:28:58.197467 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-ae941c7a-a064-4558-b911-2358c588ffb7 None None] MFA Rules not processed for user `1eae540242d94c10b0ad33960a855f08`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:28:58.198697 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-ae941c7a-a064-4558-b911-2358c588ffb7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:28:58.295083 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 656/1327] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jul 25 05:28:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:28:58.332442 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee4b4d0f-9879-43b7-956e-61b5c63a212e None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:28:58.334125 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-ee4b4d0f-9879-43b7-956e-61b5c63a212e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:28:58.439016 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee4b4d0f-9879-43b7-956e-61b5c63a212e None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:28:58.516632 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c96bfcd4-8232-42ab-adc1-7a585ccc25fb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b2931200f99e43d79a50a9f6e02738e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92afba266cce4d1bad4134c351e74be3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:28:58.517326 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96bfcd4-8232-42ab-adc1-7a585ccc25fb service neutron] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:28:58.517615 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96bfcd4-8232-42ab-adc1-7a585ccc25fb service neutron] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:28:58.517895 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96bfcd4-8232-42ab-adc1-7a585ccc25fb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:28:58.519548 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-c96bfcd4-8232-42ab-adc1-7a585ccc25fb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:28:58.537039 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:28:58.629500 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c96bfcd4-8232-42ab-adc1-7a585ccc25fb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:28:58.630971 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c96bfcd4-8232-42ab-adc1-7a585ccc25fb service neutron] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:28:58.649596 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 672/1328] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jul 25 05:28:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 05:29:00.767746 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c45edc-4f26-453c-9803-6bdefdf0dd4a None None] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:29:00.768276 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c45edc-4f26-453c-9803-6bdefdf0dd4a None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:29:00.768367 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c45edc-4f26-453c-9803-6bdefdf0dd4a None None] PATH_INFO: `/` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:29:00.769761 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 657/1329] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jul 25 05:29:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:29:00.775393 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c1bd17-8481-4fe8-b79c-65c9624c5f56 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:29:00.775873 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c1bd17-8481-4fe8-b79c-65c9624c5f56 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:29:00.775963 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c1bd17-8481-4fe8-b79c-65c9624c5f56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:29:00.845050 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-92c1bd17-8481-4fe8-b79c-65c9624c5f56 None None] MFA Rules not processed for user `c25734c1f07644de9355e54cfd5f65cc`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:29:00.846258 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-92c1bd17-8481-4fe8-b79c-65c9624c5f56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:29:00.913374 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:29:00.944869 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 673/1330] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jul 25 05:29:00 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:29:00.982269 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c45edc-4f26-453c-9803-6bdefdf0dd4a None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:29:00.983794 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-c0c45edc-4f26-453c-9803-6bdefdf0dd4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:29:01.088279 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c45edc-4f26-453c-9803-6bdefdf0dd4a None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:29:01.108993 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:29:01.163342 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab097c9-2a68-4835-8aed-9000d96b6cf0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b2931200f99e43d79a50a9f6e02738e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92afba266cce4d1bad4134c351e74be3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:29:01.164004 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab097c9-2a68-4835-8aed-9000d96b6cf0 service neutron] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:29:01.164290 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab097c9-2a68-4835-8aed-9000d96b6cf0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:29:01.164570 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab097c9-2a68-4835-8aed-9000d96b6cf0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:29:01.166253 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-dab097c9-2a68-4835-8aed-9000d96b6cf0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:29:01.288289 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dab097c9-2a68-4835-8aed-9000d96b6cf0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:29:01.290220 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dab097c9-2a68-4835-8aed-9000d96b6cf0 service neutron] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:29:01.309620 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 658/1331] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jul 25 05:29:00 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 05:29:03.641379 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c034dae6-afa6-4202-81f1-3e62ecdad812 None None] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:29:03.641962 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c034dae6-afa6-4202-81f1-3e62ecdad812 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:29:03.641962 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c034dae6-afa6-4202-81f1-3e62ecdad812 None None] PATH_INFO: `/` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:29:03.643252 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 674/1332] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jul 25 05:29:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:29:03.648349 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c093d2c-9806-4304-b4c4-382df0188285 None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:29:03.648817 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c093d2c-9806-4304-b4c4-382df0188285 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:29:03.648953 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c093d2c-9806-4304-b4c4-382df0188285 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:29:03.711753 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:29:03.714681 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-8c093d2c-9806-4304-b4c4-382df0188285 None None] MFA Rules not processed for user `1eae540242d94c10b0ad33960a855f08`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:29:03.716456 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-8c093d2c-9806-4304-b4c4-382df0188285 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:29:03.815902 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 659/1333] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jul 25 05:29:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:29:03.852885 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c034dae6-afa6-4202-81f1-3e62ecdad812 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:29:03.854408 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-c034dae6-afa6-4202-81f1-3e62ecdad812 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:29:03.957417 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c034dae6-afa6-4202-81f1-3e62ecdad812 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:29:04.029966 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb20cc40-3563-4857-b878-103c66e98046 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b2931200f99e43d79a50a9f6e02738e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92afba266cce4d1bad4134c351e74be3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:29:04.030627 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb20cc40-3563-4857-b878-103c66e98046 service neutron] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:29:04.030897 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb20cc40-3563-4857-b878-103c66e98046 service neutron] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:29:04.031251 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb20cc40-3563-4857-b878-103c66e98046 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:29:04.032893 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-fb20cc40-3563-4857-b878-103c66e98046 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:29:04.059274 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:29:04.139866 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb20cc40-3563-4857-b878-103c66e98046 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:29:04.141420 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb20cc40-3563-4857-b878-103c66e98046 service neutron] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:29:04.159159 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 675/1334] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jul 25 05:29:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 05:29:06.260233 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428d17fc-1985-4498-b32d-08b4855c7fe3 None None] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:29:06.260710 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428d17fc-1985-4498-b32d-08b4855c7fe3 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:29:06.260841 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428d17fc-1985-4498-b32d-08b4855c7fe3 None None] PATH_INFO: `/` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:29:06.262202 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 660/1335] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jul 25 05:29:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:29:06.267366 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b802781-d397-4489-b4c4-07ad71885df1 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:29:06.267809 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b802781-d397-4489-b4c4-07ad71885df1 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:29:06.267928 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b802781-d397-4489-b4c4-07ad71885df1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:29:06.331906 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-3b802781-d397-4489-b4c4-07ad71885df1 None None] MFA Rules not processed for user `1eae540242d94c10b0ad33960a855f08`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:29:06.333139 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-3b802781-d397-4489-b4c4-07ad71885df1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:29:06.405117 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:29:06.427832 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 676/1336] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jul 25 05:29:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:29:06.468230 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-428d17fc-1985-4498-b32d-08b4855c7fe3 None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:29:06.469792 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-428d17fc-1985-4498-b32d-08b4855c7fe3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:29:06.571832 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-428d17fc-1985-4498-b32d-08b4855c7fe3 None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:29:06.599052 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:29:06.643478 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00201591-6b12-4a23-b72c-3f52a8326205 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b2931200f99e43d79a50a9f6e02738e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92afba266cce4d1bad4134c351e74be3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:29:06.644116 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00201591-6b12-4a23-b72c-3f52a8326205 service neutron] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:29:06.644399 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00201591-6b12-4a23-b72c-3f52a8326205 service neutron] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:29:06.644673 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00201591-6b12-4a23-b72c-3f52a8326205 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:29:06.646316 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-00201591-6b12-4a23-b72c-3f52a8326205 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:29:06.752046 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00201591-6b12-4a23-b72c-3f52a8326205 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:29:06.753498 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00201591-6b12-4a23-b72c-3f52a8326205 service neutron] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:29:06.771474 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 661/1337] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jul 25 05:29:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 05:29:09.166354 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24f788a-96e4-494c-b7ee-91c9e6821ced None None] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:29:09.166894 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24f788a-96e4-494c-b7ee-91c9e6821ced None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:29:09.166894 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24f788a-96e4-494c-b7ee-91c9e6821ced None None] PATH_INFO: `/` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:29:09.168211 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 677/1338] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jul 25 05:29:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:29:09.173503 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce24205-a170-4b1e-adf1-d57aca4a0ce1 None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:29:09.173956 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce24205-a170-4b1e-adf1-d57aca4a0ce1 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:29:09.174065 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce24205-a170-4b1e-adf1-d57aca4a0ce1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:29:09.238790 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-9ce24205-a170-4b1e-adf1-d57aca4a0ce1 None None] MFA Rules not processed for user `1eae540242d94c10b0ad33960a855f08`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:29:09.239998 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-9ce24205-a170-4b1e-adf1-d57aca4a0ce1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:29:09.248189 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:29:09.339437 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 662/1339] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jul 25 05:29:09 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:29:09.375715 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c24f788a-96e4-494c-b7ee-91c9e6821ced None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:29:09.377287 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-c24f788a-96e4-494c-b7ee-91c9e6821ced None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:29:09.479406 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c24f788a-96e4-494c-b7ee-91c9e6821ced None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:29:09.551862 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e622a06-c6f4-4f17-82a3-f1303a69b5a5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b2931200f99e43d79a50a9f6e02738e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92afba266cce4d1bad4134c351e74be3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:29:09.552524 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e622a06-c6f4-4f17-82a3-f1303a69b5a5 service neutron] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:29:09.552842 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e622a06-c6f4-4f17-82a3-f1303a69b5a5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:29:09.553129 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e622a06-c6f4-4f17-82a3-f1303a69b5a5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:29:09.554793 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-7e622a06-c6f4-4f17-82a3-f1303a69b5a5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:29:09.598701 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:29:09.673530 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e622a06-c6f4-4f17-82a3-f1303a69b5a5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:29:09.674969 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e622a06-c6f4-4f17-82a3-f1303a69b5a5 service neutron] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:29:09.693372 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 678/1340] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jul 25 05:29:09 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 05:29:13.752403 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ad4a8b-3d57-4296-9a50-a130812030a1 None None] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:29:13.753014 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ad4a8b-3d57-4296-9a50-a130812030a1 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:29:13.753014 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ad4a8b-3d57-4296-9a50-a130812030a1 None None] PATH_INFO: `/` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:29:13.754366 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 663/1341] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jul 25 05:29:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:29:13.759614 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2e2f4e-00aa-4aea-97b8-1dcc8b279429 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:29:13.760068 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2e2f4e-00aa-4aea-97b8-1dcc8b279429 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:29:13.760164 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2e2f4e-00aa-4aea-97b8-1dcc8b279429 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:29:13.829804 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-ab2e2f4e-00aa-4aea-97b8-1dcc8b279429 None None] MFA Rules not processed for user `1eae540242d94c10b0ad33960a855f08`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:29:13.831002 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-ab2e2f4e-00aa-4aea-97b8-1dcc8b279429 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:29:13.915513 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:29:13.927558 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 679/1342] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jul 25 05:29:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:29:13.967322 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ad4a8b-3d57-4296-9a50-a130812030a1 None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:29:13.968922 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-c1ad4a8b-3d57-4296-9a50-a130812030a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:29:14.075322 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ad4a8b-3d57-4296-9a50-a130812030a1 None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:29:14.114073 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:29:14.150703 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb4bd26-ce08-45db-9538-499199017af8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b2931200f99e43d79a50a9f6e02738e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92afba266cce4d1bad4134c351e74be3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:29:14.151360 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb4bd26-ce08-45db-9538-499199017af8 service neutron] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:29:14.151643 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb4bd26-ce08-45db-9538-499199017af8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:29:14.151923 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb4bd26-ce08-45db-9538-499199017af8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:29:14.153592 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-afb4bd26-ce08-45db-9538-499199017af8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:29:14.263989 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afb4bd26-ce08-45db-9538-499199017af8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:29:14.265463 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afb4bd26-ce08-45db-9538-499199017af8 service neutron] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:29:14.284716 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 664/1343] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jul 25 05:29:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 05:29:17.051175 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e48ba19-9817-4352-a770-5a4615fdb8df None None] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:29:17.051714 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e48ba19-9817-4352-a770-5a4615fdb8df None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:29:17.051714 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e48ba19-9817-4352-a770-5a4615fdb8df None None] PATH_INFO: `/` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:29:17.053149 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 680/1344] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jul 25 05:29:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:29:17.058541 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b510c92-045f-44f0-8416-0d75d8e3f6cf None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:29:17.058963 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b510c92-045f-44f0-8416-0d75d8e3f6cf None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:29:17.059070 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b510c92-045f-44f0-8416-0d75d8e3f6cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:29:17.124957 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-3b510c92-045f-44f0-8416-0d75d8e3f6cf None None] MFA Rules not processed for user `1eae540242d94c10b0ad33960a855f08`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:29:17.126830 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-3b510c92-045f-44f0-8416-0d75d8e3f6cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:29:17.144663 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:29:17.223014 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 665/1345] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jul 25 05:29:17 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:29:17.260646 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e48ba19-9817-4352-a770-5a4615fdb8df None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:29:17.262212 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-1e48ba19-9817-4352-a770-5a4615fdb8df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:29:17.366669 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e48ba19-9817-4352-a770-5a4615fdb8df None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:29:17.441901 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45102dec-8947-4fc5-9519-d02d5115ffb6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b2931200f99e43d79a50a9f6e02738e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92afba266cce4d1bad4134c351e74be3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:29:17.442552 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45102dec-8947-4fc5-9519-d02d5115ffb6 service neutron] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:29:17.442834 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45102dec-8947-4fc5-9519-d02d5115ffb6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:29:17.443107 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45102dec-8947-4fc5-9519-d02d5115ffb6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:29:17.444750 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-45102dec-8947-4fc5-9519-d02d5115ffb6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:29:17.488646 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:29:17.557319 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45102dec-8947-4fc5-9519-d02d5115ffb6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:29:17.558757 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45102dec-8947-4fc5-9519-d02d5115ffb6 service neutron] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:29:17.581560 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 681/1346] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jul 25 05:29:17 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 05:29:21.879716 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4195a1fb-d669-42d3-9aa2-b5cf2de87422 None None] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:29:21.880286 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4195a1fb-d669-42d3-9aa2-b5cf2de87422 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:29:21.880286 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4195a1fb-d669-42d3-9aa2-b5cf2de87422 None None] PATH_INFO: `/` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:29:21.881845 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 666/1347] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jul 25 05:29:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:29:21.887601 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f38038a-122d-4c7b-85f4-2881d78b49c9 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:29:21.888088 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f38038a-122d-4c7b-85f4-2881d78b49c9 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:29:21.888088 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f38038a-122d-4c7b-85f4-2881d78b49c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:29:21.957632 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-0f38038a-122d-4c7b-85f4-2881d78b49c9 None None] MFA Rules not processed for user `1eae540242d94c10b0ad33960a855f08`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:29:21.958866 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-0f38038a-122d-4c7b-85f4-2881d78b49c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:29:22.055117 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:29:22.060950 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 682/1348] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jul 25 05:29:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:29:22.097641 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4195a1fb-d669-42d3-9aa2-b5cf2de87422 None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:29:22.099320 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-4195a1fb-d669-42d3-9aa2-b5cf2de87422 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:29:22.213979 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4195a1fb-d669-42d3-9aa2-b5cf2de87422 None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:29:22.263437 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:29:22.293712 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c29fe9ea-7e3f-434f-af6d-72a0a1ef5a93 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b2931200f99e43d79a50a9f6e02738e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92afba266cce4d1bad4134c351e74be3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:29:22.294376 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29fe9ea-7e3f-434f-af6d-72a0a1ef5a93 service neutron] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:29:22.294660 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29fe9ea-7e3f-434f-af6d-72a0a1ef5a93 service neutron] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:29:22.294934 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29fe9ea-7e3f-434f-af6d-72a0a1ef5a93 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:29:22.296619 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-c29fe9ea-7e3f-434f-af6d-72a0a1ef5a93 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:29:22.416232 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c29fe9ea-7e3f-434f-af6d-72a0a1ef5a93 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:29:22.417716 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c29fe9ea-7e3f-434f-af6d-72a0a1ef5a93 service neutron] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:29:22.436872 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 667/1349] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jul 25 05:29:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 340 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 05:29:27.318176 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816b42e6-22d6-4394-9565-4796ccb1bdc9 None None] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:29:27.318722 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816b42e6-22d6-4394-9565-4796ccb1bdc9 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:29:27.318722 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816b42e6-22d6-4394-9565-4796ccb1bdc9 None None] PATH_INFO: `/` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:29:27.320327 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 683/1350] 10.222.0.33 () {54 vars in 921 bytes} [Mon Jul 25 05:29:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:29:27.326007 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab215aff-8665-4b1f-b985-86d5fa8de184 None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:29:27.326007 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab215aff-8665-4b1f-b985-86d5fa8de184 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:29:27.326007 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab215aff-8665-4b1f-b985-86d5fa8de184 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:29:27.443607 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-ab215aff-8665-4b1f-b985-86d5fa8de184 None None] MFA Rules not processed for user `317790016e7047fe96af73925de5d872`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:29:27.445240 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-ab215aff-8665-4b1f-b985-86d5fa8de184 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:29:27.466349 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:29:27.565013 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 668/1351] 10.222.0.33 () {58 vars in 1018 bytes} [Mon Jul 25 05:29:27 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 241 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:29:31.943103 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65610e7-cfe3-4313-b1d0-e76a0a7b11f9 None None] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:29:31.943630 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65610e7-cfe3-4313-b1d0-e76a0a7b11f9 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:29:31.943630 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65610e7-cfe3-4313-b1d0-e76a0a7b11f9 None None] PATH_INFO: `/` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:29:31.945233 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 684/1352] 10.222.0.33 () {54 vars in 921 bytes} [Mon Jul 25 05:29:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:29:31.950289 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb4b582-9b08-4489-af4a-ef8219bd3344 None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:29:31.950742 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb4b582-9b08-4489-af4a-ef8219bd3344 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:29:31.950891 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb4b582-9b08-4489-af4a-ef8219bd3344 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:29:32.029781 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-6bb4b582-9b08-4489-af4a-ef8219bd3344 None None] MFA Rules not processed for user `317790016e7047fe96af73925de5d872`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:29:32.031008 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-6bb4b582-9b08-4489-af4a-ef8219bd3344 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:29:32.106508 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:29:32.146355 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 669/1353] 10.222.0.33 () {58 vars in 1018 bytes} [Mon Jul 25 05:29:31 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:29:33.168506 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65b6856-df83-4e6c-9f19-4ed6ac5e8d81 None None] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:29:33.169053 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65b6856-df83-4e6c-9f19-4ed6ac5e8d81 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:29:33.169543 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65b6856-df83-4e6c-9f19-4ed6ac5e8d81 None None] PATH_INFO: `/` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:29:33.170959 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 685/1354] 10.222.0.33 () {54 vars in 921 bytes} [Mon Jul 25 05:29:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:29:33.176607 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a03cf0-0fd9-4d3a-8f75-1901bac03f2a None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:29:33.177093 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a03cf0-0fd9-4d3a-8f75-1901bac03f2a None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:29:33.177193 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a03cf0-0fd9-4d3a-8f75-1901bac03f2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:29:33.254620 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-87a03cf0-0fd9-4d3a-8f75-1901bac03f2a None None] MFA Rules not processed for user `317790016e7047fe96af73925de5d872`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:29:33.255840 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-87a03cf0-0fd9-4d3a-8f75-1901bac03f2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:29:33.364717 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 670/1355] 10.222.0.33 () {58 vars in 1018 bytes} [Mon Jul 25 05:29:33 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:29:36.842328 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf75499-61d9-49ba-b914-93aa2d7f0d91 None None] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:29:36.843273 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf75499-61d9-49ba-b914-93aa2d7f0d91 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:29:36.843824 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf75499-61d9-49ba-b914-93aa2d7f0d91 None None] PATH_INFO: `/` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:29:36.846338 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 686/1356] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jul 25 05:29:36 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:29:36.852140 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a97ba6-39d7-4b3b-8845-32e824d64f8e None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:29:36.853138 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a97ba6-39d7-4b3b-8845-32e824d64f8e None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:29:36.853610 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a97ba6-39d7-4b3b-8845-32e824d64f8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:29:36.890796 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:29:36.928464 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-55a97ba6-39d7-4b3b-8845-32e824d64f8e None None] MFA Rules not processed for user `1eae540242d94c10b0ad33960a855f08`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:29:36.929702 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-55a97ba6-39d7-4b3b-8845-32e824d64f8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:29:37.031906 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 671/1357] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jul 25 05:29:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:29:37.089725 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451fc8b1-d5b4-4950-842f-3d80ec6b3264 None None] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:29:37.090759 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451fc8b1-d5b4-4950-842f-3d80ec6b3264 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:29:37.091475 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451fc8b1-d5b4-4950-842f-3d80ec6b3264 None None] PATH_INFO: `/` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:29:37.093763 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 687/1358] 10.222.0.33 () {54 vars in 962 bytes} [Mon Jul 25 05:29:37 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:29:37.101989 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c333f25-e4a6-4222-a978-921b4f34bfd3 None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:29:37.102279 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c333f25-e4a6-4222-a978-921b4f34bfd3 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:29:37.102558 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c333f25-e4a6-4222-a978-921b4f34bfd3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:29:37.185378 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-1c333f25-e4a6-4222-a978-921b4f34bfd3 None None] MFA Rules not processed for user `1cd482a61e8a420db45bb532e5dae6e2`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:29:37.186597 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-1c333f25-e4a6-4222-a978-921b4f34bfd3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:29:37.238199 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:29:37.312602 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 672/1359] 10.222.0.33 () {58 vars in 1059 bytes} [Mon Jul 25 05:29:37 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:29:37.320423 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-451fc8b1-d5b4-4950-842f-3d80ec6b3264 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:29:37.322069 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-451fc8b1-d5b4-4950-842f-3d80ec6b3264 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:29:37.475203 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-451fc8b1-d5b4-4950-842f-3d80ec6b3264 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:29:37.550175 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:29:37.592908 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-424fe598-4c8b-47ae-ade0-b4b8ad4d7ffe service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cd482a61e8a420db45bb532e5dae6e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92afba266cce4d1bad4134c351e74be3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:29:37.593644 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424fe598-4c8b-47ae-ade0-b4b8ad4d7ffe service nova] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:29:37.593820 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424fe598-4c8b-47ae-ade0-b4b8ad4d7ffe service nova] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:29:37.594076 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424fe598-4c8b-47ae-ade0-b4b8ad4d7ffe service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:29:37.596177 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-424fe598-4c8b-47ae-ade0-b4b8ad4d7ffe service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:29:37.707879 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-424fe598-4c8b-47ae-ade0-b4b8ad4d7ffe service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:29:37.710281 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-424fe598-4c8b-47ae-ade0-b4b8ad4d7ffe service nova] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:29:37.730567 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 688/1360] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jul 25 05:29:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 410 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 05:29:40.020758 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7994bf-69f7-4d98-8600-d6a8333454b8 None None] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:29:40.021292 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7994bf-69f7-4d98-8600-d6a8333454b8 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:29:40.021292 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7994bf-69f7-4d98-8600-d6a8333454b8 None None] PATH_INFO: `/` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:29:40.022823 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 673/1361] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jul 25 05:29:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:29:40.028442 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48cd21d4-ce05-449d-b510-d6d9405652fd None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:29:40.028948 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48cd21d4-ce05-449d-b510-d6d9405652fd None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:29:40.029034 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48cd21d4-ce05-449d-b510-d6d9405652fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:29:40.101218 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-48cd21d4-ce05-449d-b510-d6d9405652fd None None] MFA Rules not processed for user `1eae540242d94c10b0ad33960a855f08`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:29:40.102427 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-48cd21d4-ce05-449d-b510-d6d9405652fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:29:40.142447 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:29:40.213063 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 689/1362] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jul 25 05:29:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:29:40.262761 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da7994bf-69f7-4d98-8600-d6a8333454b8 None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:29:40.262761 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-da7994bf-69f7-4d98-8600-d6a8333454b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:29:40.449954 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da7994bf-69f7-4d98-8600-d6a8333454b8 None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:29:40.463308 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:29:40.580245 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db0ab5a-2e8e-4072-bcf7-dbed70bc6dcc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cd482a61e8a420db45bb532e5dae6e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92afba266cce4d1bad4134c351e74be3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:29:40.581263 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db0ab5a-2e8e-4072-bcf7-dbed70bc6dcc service nova] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:29:40.581610 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db0ab5a-2e8e-4072-bcf7-dbed70bc6dcc service nova] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:29:40.581900 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db0ab5a-2e8e-4072-bcf7-dbed70bc6dcc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:29:40.583597 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-2db0ab5a-2e8e-4072-bcf7-dbed70bc6dcc service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:29:40.677849 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2db0ab5a-2e8e-4072-bcf7-dbed70bc6dcc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:29:40.679440 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2db0ab5a-2e8e-4072-bcf7-dbed70bc6dcc service nova] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:29:40.698473 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 674/1363] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jul 25 05:29:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 440 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 05:29:43.120699 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787014f5-36ae-49a3-95a4-2b7b913ba847 None None] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:29:43.121305 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787014f5-36ae-49a3-95a4-2b7b913ba847 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:29:43.121305 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787014f5-36ae-49a3-95a4-2b7b913ba847 None None] PATH_INFO: `/` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:29:43.122604 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 690/1364] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jul 25 05:29:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:29:43.128248 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9ca26b-28f9-4494-8b1e-a9ce539421ca None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:29:43.128726 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9ca26b-28f9-4494-8b1e-a9ce539421ca None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:29:43.128899 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9ca26b-28f9-4494-8b1e-a9ce539421ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:29:43.149657 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:29:43.205535 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-4f9ca26b-28f9-4494-8b1e-a9ce539421ca None None] MFA Rules not processed for user `1eae540242d94c10b0ad33960a855f08`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:29:43.206991 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-4f9ca26b-28f9-4494-8b1e-a9ce539421ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:29:43.310685 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 675/1365] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jul 25 05:29:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:29:43.358929 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f3c410-c65f-47b2-8fcb-94706840240c None None] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:29:43.359234 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f3c410-c65f-47b2-8fcb-94706840240c None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:29:43.359523 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f3c410-c65f-47b2-8fcb-94706840240c None None] PATH_INFO: `/` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:29:43.361507 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 691/1366] 10.222.0.33 () {54 vars in 962 bytes} [Mon Jul 25 05:29:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:29:43.367132 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6d3988-4b69-409f-b2fd-77cd5aed620f None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:29:43.367497 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6d3988-4b69-409f-b2fd-77cd5aed620f None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:29:43.367850 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6d3988-4b69-409f-b2fd-77cd5aed620f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:29:43.448325 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-8f6d3988-4b69-409f-b2fd-77cd5aed620f None None] MFA Rules not processed for user `1cd482a61e8a420db45bb532e5dae6e2`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:29:43.449508 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-8f6d3988-4b69-409f-b2fd-77cd5aed620f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:29:43.486581 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:29:43.575075 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 676/1367] 10.222.0.33 () {58 vars in 1059 bytes} [Mon Jul 25 05:29:43 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:29:43.584927 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f3c410-c65f-47b2-8fcb-94706840240c None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:29:43.586485 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-48f3c410-c65f-47b2-8fcb-94706840240c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:29:43.764233 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f3c410-c65f-47b2-8fcb-94706840240c None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:29:43.785991 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:29:43.890055 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6750f9b1-61e2-47c5-9b71-09184e8b05cd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cd482a61e8a420db45bb532e5dae6e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92afba266cce4d1bad4134c351e74be3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:29:43.890715 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6750f9b1-61e2-47c5-9b71-09184e8b05cd service nova] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:29:43.891006 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6750f9b1-61e2-47c5-9b71-09184e8b05cd service nova] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:29:43.891286 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6750f9b1-61e2-47c5-9b71-09184e8b05cd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:29:43.893523 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-6750f9b1-61e2-47c5-9b71-09184e8b05cd service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:29:43.988744 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6750f9b1-61e2-47c5-9b71-09184e8b05cd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:29:43.990393 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6750f9b1-61e2-47c5-9b71-09184e8b05cd service nova] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:29:44.013675 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 692/1368] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jul 25 05:29:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 429 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 05:29:45.036493 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16fd4c6-087c-4295-9c8a-c55aff51e691 None None] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:29:45.037050 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16fd4c6-087c-4295-9c8a-c55aff51e691 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:29:45.037050 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16fd4c6-087c-4295-9c8a-c55aff51e691 None None] PATH_INFO: `/` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:29:45.038565 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 677/1369] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jul 25 05:29:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:29:45.043635 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5b993e-02fb-4a76-b082-481168dc2ec0 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:29:45.044040 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5b993e-02fb-4a76-b082-481168dc2ec0 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:29:45.044211 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5b993e-02fb-4a76-b082-481168dc2ec0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:29:45.075732 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:29:45.113466 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-2c5b993e-02fb-4a76-b082-481168dc2ec0 None None] MFA Rules not processed for user `1eae540242d94c10b0ad33960a855f08`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:29:45.114695 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-2c5b993e-02fb-4a76-b082-481168dc2ec0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:29:45.217304 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 693/1370] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jul 25 05:29:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:29:45.570160 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5eddaa5-6d09-4457-bad0-23faad7644e0 None None] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:29:45.570160 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5eddaa5-6d09-4457-bad0-23faad7644e0 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:29:45.570160 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5eddaa5-6d09-4457-bad0-23faad7644e0 None None] PATH_INFO: `/` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:29:45.570160 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 678/1371] 10.222.0.33 () {54 vars in 964 bytes} [Mon Jul 25 05:29:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 25 05:29:45.574160 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69adc6a3-5392-4a28-97a3-8127ea209f12 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:29:45.574160 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69adc6a3-5392-4a28-97a3-8127ea209f12 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:29:45.574160 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69adc6a3-5392-4a28-97a3-8127ea209f12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:29:45.671048 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-69adc6a3-5392-4a28-97a3-8127ea209f12 None None] MFA Rules not processed for user `db57b478c25643dbaf936f90f6e17d0a`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:29:45.672874 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-69adc6a3-5392-4a28-97a3-8127ea209f12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:29:45.733608 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:29:45.774869 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 694/1372] 10.222.0.33 () {58 vars in 1061 bytes} [Mon Jul 25 05:29:45 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:29:45.785808 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5eddaa5-6d09-4457-bad0-23faad7644e0 None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:29:45.787312 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-b5eddaa5-6d09-4457-bad0-23faad7644e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:29:45.929504 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5eddaa5-6d09-4457-bad0-23faad7644e0 None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:29:45.978662 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:29:46.015002 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3545b46-ab5c-4705-80bc-36d5f10a234f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db57b478c25643dbaf936f90f6e17d0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92afba266cce4d1bad4134c351e74be3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:29:46.016608 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3545b46-ab5c-4705-80bc-36d5f10a234f service cinder] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:29:46.017499 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3545b46-ab5c-4705-80bc-36d5f10a234f service cinder] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:29:46.017782 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3545b46-ab5c-4705-80bc-36d5f10a234f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:29:46.019553 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-a3545b46-ab5c-4705-80bc-36d5f10a234f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:29:46.136650 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3545b46-ab5c-4705-80bc-36d5f10a234f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:29:46.138172 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3545b46-ab5c-4705-80bc-36d5f10a234f service cinder] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:29:46.158035 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 679/1373] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jul 25 05:29:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 374 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 05:29:46.196393 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0dc2372-34f3-43d4-bf00-21f0b24075ec None None] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:29:46.196926 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0dc2372-34f3-43d4-bf00-21f0b24075ec None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:29:46.196926 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0dc2372-34f3-43d4-bf00-21f0b24075ec None None] PATH_INFO: `/` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:29:46.198586 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 695/1374] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jul 25 05:29:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:29:46.203360 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de999a4c-4c60-4459-ba21-faa4e7283277 None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:29:46.203709 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de999a4c-4c60-4459-ba21-faa4e7283277 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:29:46.204013 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de999a4c-4c60-4459-ba21-faa4e7283277 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:29:46.271970 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-de999a4c-4c60-4459-ba21-faa4e7283277 None None] MFA Rules not processed for user `1eae540242d94c10b0ad33960a855f08`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:29:46.273329 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-de999a4c-4c60-4459-ba21-faa4e7283277 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:29:46.294740 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:29:46.380094 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 680/1375] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jul 25 05:29:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:29:46.429317 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0dc2372-34f3-43d4-bf00-21f0b24075ec None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:29:46.429317 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-a0dc2372-34f3-43d4-bf00-21f0b24075ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:29:46.589079 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0dc2372-34f3-43d4-bf00-21f0b24075ec None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:29:46.641600 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:29:46.703992 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c260ab3-080e-45d5-a0c7-2d2581b273df service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cd482a61e8a420db45bb532e5dae6e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92afba266cce4d1bad4134c351e74be3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:29:46.705949 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c260ab3-080e-45d5-a0c7-2d2581b273df service nova] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:29:46.706234 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c260ab3-080e-45d5-a0c7-2d2581b273df service nova] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:29:46.706509 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c260ab3-080e-45d5-a0c7-2d2581b273df service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:29:46.708167 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-0c260ab3-080e-45d5-a0c7-2d2581b273df service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:29:46.796918 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c260ab3-080e-45d5-a0c7-2d2581b273df service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:29:46.799356 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c260ab3-080e-45d5-a0c7-2d2581b273df service nova] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:29:46.822350 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 696/1376] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jul 25 05:29:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 396 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 05:29:48.888986 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85be0ba2-f87e-464b-a04c-f31e154fb10e None None] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:29:48.889527 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85be0ba2-f87e-464b-a04c-f31e154fb10e None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:29:48.889527 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85be0ba2-f87e-464b-a04c-f31e154fb10e None None] PATH_INFO: `/` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:29:48.890971 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 681/1377] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jul 25 05:29:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:29:48.896381 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8797fff4-58ce-42c2-9ada-e722321ecd4d None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:29:48.896874 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8797fff4-58ce-42c2-9ada-e722321ecd4d None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:29:48.896991 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8797fff4-58ce-42c2-9ada-e722321ecd4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:29:48.970710 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-8797fff4-58ce-42c2-9ada-e722321ecd4d None None] MFA Rules not processed for user `1eae540242d94c10b0ad33960a855f08`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:29:48.971964 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-8797fff4-58ce-42c2-9ada-e722321ecd4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:29:48.978864 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:29:49.077404 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 697/1378] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jul 25 05:29:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:29:49.130418 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85be0ba2-f87e-464b-a04c-f31e154fb10e None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:29:49.131938 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-85be0ba2-f87e-464b-a04c-f31e154fb10e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:29:49.278252 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85be0ba2-f87e-464b-a04c-f31e154fb10e None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:29:49.281743 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:29:49.398788 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b67f81-1e6e-415d-8e6c-55e955e91ac5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cd482a61e8a420db45bb532e5dae6e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92afba266cce4d1bad4134c351e74be3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:29:49.399336 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b67f81-1e6e-415d-8e6c-55e955e91ac5 service nova] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:29:49.399618 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b67f81-1e6e-415d-8e6c-55e955e91ac5 service nova] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:29:49.399889 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b67f81-1e6e-415d-8e6c-55e955e91ac5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:29:49.401572 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-e6b67f81-1e6e-415d-8e6c-55e955e91ac5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:29:49.504946 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6b67f81-1e6e-415d-8e6c-55e955e91ac5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:29:49.506535 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6b67f81-1e6e-415d-8e6c-55e955e91ac5 service nova] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:29:49.526457 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 682/1379] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jul 25 05:29:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 396 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 05:29:51.663413 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cdcd306-80cf-431a-9811-992e51a2833a None None] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:29:51.664088 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cdcd306-80cf-431a-9811-992e51a2833a None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:29:51.664088 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cdcd306-80cf-431a-9811-992e51a2833a None None] PATH_INFO: `/` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:29:51.665874 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 698/1380] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jul 25 05:29:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:29:51.671369 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be2abe8-7078-4f01-855a-317ee4332509 None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:29:51.671894 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be2abe8-7078-4f01-855a-317ee4332509 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:29:51.672295 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be2abe8-7078-4f01-855a-317ee4332509 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:29:51.685576 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:29:51.761162 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-8be2abe8-7078-4f01-855a-317ee4332509 None None] MFA Rules not processed for user `1eae540242d94c10b0ad33960a855f08`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:29:51.762546 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-8be2abe8-7078-4f01-855a-317ee4332509 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:29:51.900892 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 683/1381] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jul 25 05:29:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 228 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:29:51.943376 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cdcd306-80cf-431a-9811-992e51a2833a None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:29:51.944928 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-5cdcd306-80cf-431a-9811-992e51a2833a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:29:52.093148 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:29:52.105752 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cdcd306-80cf-431a-9811-992e51a2833a None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:29:52.228810 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-902d0d39-1f49-41a3-a342-4920d9bfeab2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cd482a61e8a420db45bb532e5dae6e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92afba266cce4d1bad4134c351e74be3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:29:52.229424 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902d0d39-1f49-41a3-a342-4920d9bfeab2 service nova] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:29:52.229816 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902d0d39-1f49-41a3-a342-4920d9bfeab2 service nova] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:29:52.230102 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902d0d39-1f49-41a3-a342-4920d9bfeab2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:29:52.231758 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-902d0d39-1f49-41a3-a342-4920d9bfeab2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:29:52.328549 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-902d0d39-1f49-41a3-a342-4920d9bfeab2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:29:52.331008 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-902d0d39-1f49-41a3-a342-4920d9bfeab2 service nova] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:29:52.341998 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:29:52.352953 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 699/1382] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jul 25 05:29:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 410 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 05:29:54.446479 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e822d5ce-2857-4f34-bad0-f4f79b5f6006 None None] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:29:54.447044 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e822d5ce-2857-4f34-bad0-f4f79b5f6006 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:29:54.447206 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e822d5ce-2857-4f34-bad0-f4f79b5f6006 None None] PATH_INFO: `/` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:29:54.448874 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 684/1383] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jul 25 05:29:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:29:54.455464 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90755872-8f4c-42de-88b2-212f32cfb758 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:29:54.456139 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90755872-8f4c-42de-88b2-212f32cfb758 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:29:54.456387 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90755872-8f4c-42de-88b2-212f32cfb758 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:29:54.534451 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-90755872-8f4c-42de-88b2-212f32cfb758 None None] MFA Rules not processed for user `1eae540242d94c10b0ad33960a855f08`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:29:54.535988 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-90755872-8f4c-42de-88b2-212f32cfb758 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:29:54.653100 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 700/1384] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jul 25 05:29:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:29:54.697308 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e822d5ce-2857-4f34-bad0-f4f79b5f6006 None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:29:54.698843 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-e822d5ce-2857-4f34-bad0-f4f79b5f6006 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:29:54.789577 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:29:54.858287 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e822d5ce-2857-4f34-bad0-f4f79b5f6006 None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:29:54.994202 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd4bbe9-dfb9-4e9a-880b-64f3df7d7cb5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cd482a61e8a420db45bb532e5dae6e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92afba266cce4d1bad4134c351e74be3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:29:54.994856 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd4bbe9-dfb9-4e9a-880b-64f3df7d7cb5 service nova] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:29:54.995168 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd4bbe9-dfb9-4e9a-880b-64f3df7d7cb5 service nova] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:29:54.995452 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd4bbe9-dfb9-4e9a-880b-64f3df7d7cb5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:29:54.997224 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-efd4bbe9-dfb9-4e9a-880b-64f3df7d7cb5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:29:55.050733 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:29:55.098807 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efd4bbe9-dfb9-4e9a-880b-64f3df7d7cb5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:29:55.100469 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efd4bbe9-dfb9-4e9a-880b-64f3df7d7cb5 service nova] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:29:55.118725 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 685/1385] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jul 25 05:29:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 422 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 05:29:57.992445 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8cf565e-e187-4e56-9593-0098e8db3b36 None None] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:29:57.993381 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8cf565e-e187-4e56-9593-0098e8db3b36 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:29:57.993665 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8cf565e-e187-4e56-9593-0098e8db3b36 None None] PATH_INFO: `/` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:29:57.995114 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 701/1386] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jul 25 05:29:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:29:58.000520 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2f0472-d67d-46a1-8883-eed934e41e83 None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:29:58.001177 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2f0472-d67d-46a1-8883-eed934e41e83 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:29:58.001177 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2f0472-d67d-46a1-8883-eed934e41e83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:29:58.071449 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-dc2f0472-d67d-46a1-8883-eed934e41e83 None None] MFA Rules not processed for user `1eae540242d94c10b0ad33960a855f08`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:29:58.072696 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-dc2f0472-d67d-46a1-8883-eed934e41e83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:29:58.173287 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 686/1387] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jul 25 05:29:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:29:58.219297 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8cf565e-e187-4e56-9593-0098e8db3b36 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:29:58.220976 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-e8cf565e-e187-4e56-9593-0098e8db3b36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:29:58.305408 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:29:58.392267 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8cf565e-e187-4e56-9593-0098e8db3b36 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:29:58.521161 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eadf6fc-c59f-4eb6-b015-7fbced956033 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cd482a61e8a420db45bb532e5dae6e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92afba266cce4d1bad4134c351e74be3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:29:58.521695 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eadf6fc-c59f-4eb6-b015-7fbced956033 service nova] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:29:58.521967 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eadf6fc-c59f-4eb6-b015-7fbced956033 service nova] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:29:58.522247 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eadf6fc-c59f-4eb6-b015-7fbced956033 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:29:58.523908 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-7eadf6fc-c59f-4eb6-b015-7fbced956033 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:29:58.564008 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:29:58.629040 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eadf6fc-c59f-4eb6-b015-7fbced956033 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:29:58.631165 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eadf6fc-c59f-4eb6-b015-7fbced956033 service nova] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:29:58.653296 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 702/1388] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jul 25 05:29:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 434 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 05:30:00.735207 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819289b7-795a-4e4e-8f23-03ccd1e97883 None None] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:30:00.735802 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819289b7-795a-4e4e-8f23-03ccd1e97883 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:30:00.735802 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819289b7-795a-4e4e-8f23-03ccd1e97883 None None] PATH_INFO: `/` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:30:00.738122 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 687/1389] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jul 25 05:30:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:30:00.743596 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083fa19d-a5f3-4452-8cf1-b97fd5784c1e None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:30:00.744039 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083fa19d-a5f3-4452-8cf1-b97fd5784c1e None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:30:00.744158 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083fa19d-a5f3-4452-8cf1-b97fd5784c1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:30:00.809880 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-083fa19d-a5f3-4452-8cf1-b97fd5784c1e None None] MFA Rules not processed for user `1eae540242d94c10b0ad33960a855f08`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:30:00.811117 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-083fa19d-a5f3-4452-8cf1-b97fd5784c1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:30:00.907038 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 703/1390] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jul 25 05:30:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:30:00.951879 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-819289b7-795a-4e4e-8f23-03ccd1e97883 None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:30:00.953537 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-819289b7-795a-4e4e-8f23-03ccd1e97883 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:30:00.985915 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:30:01.103020 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-819289b7-795a-4e4e-8f23-03ccd1e97883 None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:30:01.204133 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:30:01.217349 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1564254-5662-4a7e-9ce6-7a0d1bee646f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cd482a61e8a420db45bb532e5dae6e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92afba266cce4d1bad4134c351e74be3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:30:01.217993 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1564254-5662-4a7e-9ce6-7a0d1bee646f service nova] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:30:01.218274 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1564254-5662-4a7e-9ce6-7a0d1bee646f service nova] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:30:01.218546 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1564254-5662-4a7e-9ce6-7a0d1bee646f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:30:01.220169 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-e1564254-5662-4a7e-9ce6-7a0d1bee646f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:30:01.310260 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1564254-5662-4a7e-9ce6-7a0d1bee646f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:30:01.311881 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1564254-5662-4a7e-9ce6-7a0d1bee646f service nova] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:30:01.330824 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 688/1391] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jul 25 05:30:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 379 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 05:30:01.390279 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90a68a5-db77-4a9e-8aa9-dd49757578e3 None None] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:30:01.390573 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90a68a5-db77-4a9e-8aa9-dd49757578e3 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:30:01.390854 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90a68a5-db77-4a9e-8aa9-dd49757578e3 None None] PATH_INFO: `/` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:30:01.392349 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 704/1392] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jul 25 05:30:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:30:01.398156 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a040c19-4f7d-4ad6-9763-b9cf58795f90 None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:30:01.398445 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a040c19-4f7d-4ad6-9763-b9cf58795f90 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:30:01.398723 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a040c19-4f7d-4ad6-9763-b9cf58795f90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:30:01.475513 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-1a040c19-4f7d-4ad6-9763-b9cf58795f90 None None] MFA Rules not processed for user `1eae540242d94c10b0ad33960a855f08`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:30:01.476968 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-1a040c19-4f7d-4ad6-9763-b9cf58795f90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:30:01.539547 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:30:01.578855 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 689/1393] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jul 25 05:30:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:30:01.636014 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b016ee8-eb43-42e1-a2dd-657b5d45856c None None] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:30:01.636403 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b016ee8-eb43-42e1-a2dd-657b5d45856c None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:30:01.636671 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b016ee8-eb43-42e1-a2dd-657b5d45856c None None] PATH_INFO: `/` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:30:01.638167 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 705/1394] 10.222.0.33 () {54 vars in 966 bytes} [Mon Jul 25 05:30:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:30:01.643300 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3924e38b-cae5-46a6-84ee-c155f32eab85 None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:30:01.643591 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3924e38b-cae5-46a6-84ee-c155f32eab85 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:30:01.643874 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3924e38b-cae5-46a6-84ee-c155f32eab85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:30:01.718229 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-3924e38b-cae5-46a6-84ee-c155f32eab85 None None] MFA Rules not processed for user `de99e44bdf9c4030adbe9c0efa3831cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:30:01.719442 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-3924e38b-cae5-46a6-84ee-c155f32eab85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:30:01.803535 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 690/1395] 10.222.0.33 () {58 vars in 1063 bytes} [Mon Jul 25 05:30:01 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:30:01.810859 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b016ee8-eb43-42e1-a2dd-657b5d45856c None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:30:01.812592 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-3b016ee8-eb43-42e1-a2dd-657b5d45856c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:30:01.827110 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:30:01.918130 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b016ee8-eb43-42e1-a2dd-657b5d45856c None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:30:01.995045 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de8cb5a2-fac9-4a28-bb02-6210c8161794 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de99e44bdf9c4030adbe9c0efa3831cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92afba266cce4d1bad4134c351e74be3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:30:01.995701 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8cb5a2-fac9-4a28-bb02-6210c8161794 service glance] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:30:01.995991 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8cb5a2-fac9-4a28-bb02-6210c8161794 service glance] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:30:01.996272 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8cb5a2-fac9-4a28-bb02-6210c8161794 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:30:01.997960 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-de8cb5a2-fac9-4a28-bb02-6210c8161794 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:30:02.067457 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:30:02.109991 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de8cb5a2-fac9-4a28-bb02-6210c8161794 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:30:02.112023 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de8cb5a2-fac9-4a28-bb02-6210c8161794 service glance] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:30:02.131748 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 706/1396] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jul 25 05:30:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 05:30:02.849057 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ffcc658-32c1-438b-b71b-97889e9a9813 None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:30:02.849535 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ffcc658-32c1-438b-b71b-97889e9a9813 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:30:02.849627 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ffcc658-32c1-438b-b71b-97889e9a9813 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:30:02.915462 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-0ffcc658-32c1-438b-b71b-97889e9a9813 None None] MFA Rules not processed for user `3352531be1b54f76b864383ccc2e5ec2`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:30:02.916674 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-0ffcc658-32c1-438b-b71b-97889e9a9813 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:30:02.925644 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:30:03.007330 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 691/1397] 10.222.0.33 () {58 vars in 1009 bytes} [Mon Jul 25 05:30:02 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:30:03.054301 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7ec3f6-a627-4806-bd1e-87cbe5ddf718 None None] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:30:03.054822 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7ec3f6-a627-4806-bd1e-87cbe5ddf718 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:30:03.054940 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7ec3f6-a627-4806-bd1e-87cbe5ddf718 None None] PATH_INFO: `/` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:30:03.056274 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 707/1398] 10.222.0.33 () {54 vars in 943 bytes} [Mon Jul 25 05:30:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:30:03.062996 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b247bf-7edd-4dfb-98a7-ae67d9ac0c8d None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:30:03.063360 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b247bf-7edd-4dfb-98a7-ae67d9ac0c8d None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:30:03.063711 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b247bf-7edd-4dfb-98a7-ae67d9ac0c8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:30:03.151159 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-17b247bf-7edd-4dfb-98a7-ae67d9ac0c8d None None] MFA Rules not processed for user `eb135a8d5d7444cfbfa336115ba2ba44`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:30:03.152378 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-17b247bf-7edd-4dfb-98a7-ae67d9ac0c8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:30:03.238860 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:30:03.261323 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 692/1399] 10.222.0.33 () {58 vars in 1040 bytes} [Mon Jul 25 05:30:03 2022] POST /identity/v3/auth/tokens => generated 3294 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:30:03.270798 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b7ec3f6-a627-4806-bd1e-87cbe5ddf718 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:30:03.272315 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-6b7ec3f6-a627-4806-bd1e-87cbe5ddf718 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:30:03.407194 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b7ec3f6-a627-4806-bd1e-87cbe5ddf718 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:30:03.445165 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:30:03.451629 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c40361-f357-4f2c-91fa-aa156546e63c None None] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:30:03.451946 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c40361-f357-4f2c-91fa-aa156546e63c None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:30:03.452224 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c40361-f357-4f2c-91fa-aa156546e63c None None] PATH_INFO: `/` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:30:03.453686 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 693/1400] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jul 25 05:30:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 25 05:30:03.459589 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7fdf83c-9c60-4a9a-9d42-071c25411258 None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:30:03.459877 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7fdf83c-9c60-4a9a-9d42-071c25411258 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:30:03.460157 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7fdf83c-9c60-4a9a-9d42-071c25411258 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:30:03.514104 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a640d7e-b4a4-49ae-a915-dc5e21dd1ecc service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb135a8d5d7444cfbfa336115ba2ba44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92afba266cce4d1bad4134c351e74be3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:30:03.514931 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a640d7e-b4a4-49ae-a915-dc5e21dd1ecc service swift] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:30:03.515307 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a640d7e-b4a4-49ae-a915-dc5e21dd1ecc service swift] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:30:03.515677 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a640d7e-b4a4-49ae-a915-dc5e21dd1ecc service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:30:03.517546 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-6a640d7e-b4a4-49ae-a915-dc5e21dd1ecc service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:30:03.525113 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-b7fdf83c-9c60-4a9a-9d42-071c25411258 None None] MFA Rules not processed for user `1eae540242d94c10b0ad33960a855f08`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:30:03.525982 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-b7fdf83c-9c60-4a9a-9d42-071c25411258 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:30:03.602805 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a640d7e-b4a4-49ae-a915-dc5e21dd1ecc service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:30:03.604429 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a640d7e-b4a4-49ae-a915-dc5e21dd1ecc service swift] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:30:03.613020 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 708/1401] 10.222.0.33 () {60 vars in 1384 bytes} [Mon Jul 25 05:30:03 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 25 05:30:03.627539 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 694/1402] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jul 25 05:30:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 05:30:03.679017 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a640d7e-b4a4-49ae-a915-dc5e21dd1ecc service swift] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:30:03.680316 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-6a640d7e-b4a4-49ae-a915-dc5e21dd1ecc service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:30:03.756965 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:30:03.790231 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a640d7e-b4a4-49ae-a915-dc5e21dd1ecc service swift] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:30:03.928526 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c47dd06-b54f-46d3-8dfe-38088433007d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cd482a61e8a420db45bb532e5dae6e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92afba266cce4d1bad4134c351e74be3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:30:03.929129 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c47dd06-b54f-46d3-8dfe-38088433007d service nova] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:30:03.929419 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c47dd06-b54f-46d3-8dfe-38088433007d service nova] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:30:03.929699 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c47dd06-b54f-46d3-8dfe-38088433007d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:30:03.931364 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-6c47dd06-b54f-46d3-8dfe-38088433007d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:30:04.037931 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:30:04.043008 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c47dd06-b54f-46d3-8dfe-38088433007d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:30:04.044642 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c47dd06-b54f-46d3-8dfe-38088433007d service nova] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:30:04.066207 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 709/1403] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jul 25 05:30:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 386 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 05:30:06.181938 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7acde4-a4df-444c-a1ca-4309282ba1f7 None None] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:30:06.182526 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7acde4-a4df-444c-a1ca-4309282ba1f7 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:30:06.182526 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7acde4-a4df-444c-a1ca-4309282ba1f7 None None] PATH_INFO: `/` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:30:06.183792 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 695/1404] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jul 25 05:30:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:30:06.189309 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848e8245-0ca2-4b77-8e4f-91663101668d None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:30:06.190902 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848e8245-0ca2-4b77-8e4f-91663101668d None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:30:06.190902 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848e8245-0ca2-4b77-8e4f-91663101668d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:30:06.261582 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-848e8245-0ca2-4b77-8e4f-91663101668d None None] MFA Rules not processed for user `1eae540242d94c10b0ad33960a855f08`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:30:06.262820 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-848e8245-0ca2-4b77-8e4f-91663101668d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:30:06.372219 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 710/1405] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jul 25 05:30:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:30:06.417276 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db7acde4-a4df-444c-a1ca-4309282ba1f7 None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:30:06.418864 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-db7acde4-a4df-444c-a1ca-4309282ba1f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:30:06.492857 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:30:06.588040 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db7acde4-a4df-444c-a1ca-4309282ba1f7 None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:30:06.714962 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c15918-a41d-4f80-bdef-03807bfe5935 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cd482a61e8a420db45bb532e5dae6e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92afba266cce4d1bad4134c351e74be3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:30:06.716032 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c15918-a41d-4f80-bdef-03807bfe5935 service nova] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:30:06.716316 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c15918-a41d-4f80-bdef-03807bfe5935 service nova] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:30:06.716587 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c15918-a41d-4f80-bdef-03807bfe5935 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:30:06.718281 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-35c15918-a41d-4f80-bdef-03807bfe5935 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:30:06.747543 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:30:06.823258 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35c15918-a41d-4f80-bdef-03807bfe5935 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:30:06.824936 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35c15918-a41d-4f80-bdef-03807bfe5935 service nova] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:30:06.845271 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 696/1406] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jul 25 05:30:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 428 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 05:30:08.956368 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af2112b-a235-442f-84ae-ff90dd90631b None None] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:30:08.957257 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af2112b-a235-442f-84ae-ff90dd90631b None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:30:08.957257 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af2112b-a235-442f-84ae-ff90dd90631b None None] PATH_INFO: `/` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:30:08.958634 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 711/1407] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jul 25 05:30:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:30:08.963921 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7ed420-6816-411d-92f6-d093b0dc4b18 None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:30:08.964361 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7ed420-6816-411d-92f6-d093b0dc4b18 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:30:08.964548 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7ed420-6816-411d-92f6-d093b0dc4b18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:30:09.037572 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-1a7ed420-6816-411d-92f6-d093b0dc4b18 None None] MFA Rules not processed for user `1eae540242d94c10b0ad33960a855f08`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:30:09.038804 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-1a7ed420-6816-411d-92f6-d093b0dc4b18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:30:09.139882 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:30:09.145923 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 697/1408] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jul 25 05:30:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:30:09.191293 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af2112b-a235-442f-84ae-ff90dd90631b None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:30:09.192845 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-4af2112b-a235-442f-84ae-ff90dd90631b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:30:09.286659 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:30:09.407641 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af2112b-a235-442f-84ae-ff90dd90631b None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:30:09.556323 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5541254c-b653-4ede-8bd4-bd603f5de14d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cd482a61e8a420db45bb532e5dae6e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92afba266cce4d1bad4134c351e74be3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:30:09.557077 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5541254c-b653-4ede-8bd4-bd603f5de14d service nova] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:30:09.557457 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5541254c-b653-4ede-8bd4-bd603f5de14d service nova] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:30:09.557979 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5541254c-b653-4ede-8bd4-bd603f5de14d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:30:09.559724 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-5541254c-b653-4ede-8bd4-bd603f5de14d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:30:09.600161 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:30:09.687983 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5541254c-b653-4ede-8bd4-bd603f5de14d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:30:09.689573 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5541254c-b653-4ede-8bd4-bd603f5de14d service nova] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:30:09.714412 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 712/1409] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jul 25 05:30:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 523 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 25 05:30:19.709357 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=34 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:30:26.294569 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac333470-db50-480a-98ac-e0d64004a583 None None] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:30:26.295022 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac333470-db50-480a-98ac-e0d64004a583 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:30:26.295136 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac333470-db50-480a-98ac-e0d64004a583 None None] PATH_INFO: `/` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:30:26.296528 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 698/1410] 10.222.0.33 () {54 vars in 918 bytes} [Mon Jul 25 05:30:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 25 05:30:26.301563 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b4a809-5e66-42ad-a103-24c27ac01393 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:30:26.301977 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b4a809-5e66-42ad-a103-24c27ac01393 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:30:26.302111 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b4a809-5e66-42ad-a103-24c27ac01393 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:30:26.376440 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-36b4a809-5e66-42ad-a103-24c27ac01393 None None] MFA Rules not processed for user `317790016e7047fe96af73925de5d872`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:30:26.377696 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-36b4a809-5e66-42ad-a103-24c27ac01393 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:30:26.483349 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 713/1411] 10.222.0.33 () {58 vars in 1015 bytes} [Mon Jul 25 05:30:26 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:30:26.497968 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11cd836-341e-486e-83d7-d1e6317cac85 None None] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:30:26.498256 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11cd836-341e-486e-83d7-d1e6317cac85 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:30:26.498533 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11cd836-341e-486e-83d7-d1e6317cac85 None None] PATH_INFO: `/` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:30:26.499891 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 699/1412] 10.222.0.33 () {54 vars in 918 bytes} [Mon Jul 25 05:30:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:30:26.505375 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9adf6005-7c9f-4a50-82d3-4d356fd07ea3 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:30:26.505667 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9adf6005-7c9f-4a50-82d3-4d356fd07ea3 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:30:26.505946 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9adf6005-7c9f-4a50-82d3-4d356fd07ea3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:30:26.576362 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:30:26.579105 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-9adf6005-7c9f-4a50-82d3-4d356fd07ea3 None None] MFA Rules not processed for user `1cd482a61e8a420db45bb532e5dae6e2`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:30:26.581032 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-9adf6005-7c9f-4a50-82d3-4d356fd07ea3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:30:26.697327 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 714/1413] 10.222.0.33 () {58 vars in 1015 bytes} [Mon Jul 25 05:30:26 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 05:30:29.238347 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48149f6b-6c3b-4b58-b245-d00bf7ebefde None None] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:30:29.238922 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48149f6b-6c3b-4b58-b245-d00bf7ebefde None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:30:29.238922 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48149f6b-6c3b-4b58-b245-d00bf7ebefde None None] PATH_INFO: `/` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:30:29.240283 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 700/1414] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jul 25 05:30:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:30:29.245696 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e01f73-28de-4913-8a83-03f6e9e6777b None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:30:29.246131 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e01f73-28de-4913-8a83-03f6e9e6777b None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:30:29.246286 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e01f73-28de-4913-8a83-03f6e9e6777b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:30:29.310409 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-60e01f73-28de-4913-8a83-03f6e9e6777b None None] MFA Rules not processed for user `1eae540242d94c10b0ad33960a855f08`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:30:29.311630 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-60e01f73-28de-4913-8a83-03f6e9e6777b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:30:29.367358 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:30:29.409974 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 715/1415] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jul 25 05:30:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:30:29.452344 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48149f6b-6c3b-4b58-b245-d00bf7ebefde None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:30:29.453890 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-48149f6b-6c3b-4b58-b245-d00bf7ebefde None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:30:29.601822 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48149f6b-6c3b-4b58-b245-d00bf7ebefde None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:30:29.674086 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:30:29.715762 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e01374d9-4a0b-4ef9-b235-20e272151721 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cd482a61e8a420db45bb532e5dae6e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92afba266cce4d1bad4134c351e74be3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:30:29.716408 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01374d9-4a0b-4ef9-b235-20e272151721 service nova] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:30:29.716689 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01374d9-4a0b-4ef9-b235-20e272151721 service nova] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:30:29.716992 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01374d9-4a0b-4ef9-b235-20e272151721 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:30:29.718638 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-e01374d9-4a0b-4ef9-b235-20e272151721 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:30:29.808392 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e01374d9-4a0b-4ef9-b235-20e272151721 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:30:29.810065 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e01374d9-4a0b-4ef9-b235-20e272151721 service nova] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:30:29.827750 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 701/1416] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jul 25 05:30:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 376 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 05:30:30.736469 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9945b6d0-f921-4f1f-9639-08ade7a81e8e None None] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:30:30.737058 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9945b6d0-f921-4f1f-9639-08ade7a81e8e None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:30:30.737058 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9945b6d0-f921-4f1f-9639-08ade7a81e8e None None] PATH_INFO: `/` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:30:30.738408 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 716/1417] 127.0.0.1 () {54 vars in 901 bytes} [Mon Jul 25 05:30:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:30:30.743483 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96206797-cb11-4740-a741-4c8cd761f2aa None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:30:30.743921 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96206797-cb11-4740-a741-4c8cd761f2aa None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:30:30.744035 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96206797-cb11-4740-a741-4c8cd761f2aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:30:30.819304 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-96206797-cb11-4740-a741-4c8cd761f2aa None None] MFA Rules not processed for user `1eae540242d94c10b0ad33960a855f08`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:30:30.820514 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-96206797-cb11-4740-a741-4c8cd761f2aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:30:30.848218 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:30:30.918761 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 702/1418] 127.0.0.1 () {58 vars in 998 bytes} [Mon Jul 25 05:30:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:30:31.069525 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9945b6d0-f921-4f1f-9639-08ade7a81e8e None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:30:31.071076 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-9945b6d0-f921-4f1f-9639-08ade7a81e8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:30:31.218847 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9945b6d0-f921-4f1f-9639-08ade7a81e8e None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:30:31.259894 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:30:31.332620 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16636228-bc13-4ee8-b5db-2ba02e25612a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cd482a61e8a420db45bb532e5dae6e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92afba266cce4d1bad4134c351e74be3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:30:31.333259 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16636228-bc13-4ee8-b5db-2ba02e25612a service nova] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:30:31.333830 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16636228-bc13-4ee8-b5db-2ba02e25612a service nova] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:30:31.334111 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16636228-bc13-4ee8-b5db-2ba02e25612a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:30:31.335770 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-16636228-bc13-4ee8-b5db-2ba02e25612a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:30:31.423972 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16636228-bc13-4ee8-b5db-2ba02e25612a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:30:31.426380 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16636228-bc13-4ee8-b5db-2ba02e25612a service nova] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:30:31.444074 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 717/1419] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jul 25 05:30:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 375 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 05:30:32.340476 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41c7cd8-7317-4a13-b1f9-a668410c0554 None None] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:30:32.341007 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41c7cd8-7317-4a13-b1f9-a668410c0554 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:30:32.341105 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41c7cd8-7317-4a13-b1f9-a668410c0554 None None] PATH_INFO: `/` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:30:32.342434 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 703/1420] 127.0.0.1 () {54 vars in 901 bytes} [Mon Jul 25 05:30:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:30:32.347463 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea44fee-ff04-4c1d-833a-7d6956915b6d None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:30:32.347901 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea44fee-ff04-4c1d-833a-7d6956915b6d None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:30:32.348011 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea44fee-ff04-4c1d-833a-7d6956915b6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:30:32.400692 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:30:32.423185 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-9ea44fee-ff04-4c1d-833a-7d6956915b6d None None] MFA Rules not processed for user `1eae540242d94c10b0ad33960a855f08`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:30:32.424388 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-9ea44fee-ff04-4c1d-833a-7d6956915b6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:30:32.522509 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 718/1421] 127.0.0.1 () {58 vars in 998 bytes} [Mon Jul 25 05:30:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:30:32.686434 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c41c7cd8-7317-4a13-b1f9-a668410c0554 None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:30:32.687897 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-c41c7cd8-7317-4a13-b1f9-a668410c0554 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:30:32.838046 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c41c7cd8-7317-4a13-b1f9-a668410c0554 None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:30:32.849908 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:30:32.953310 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d77a2796-8ccb-4ae6-9054-14973aa0c428 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cd482a61e8a420db45bb532e5dae6e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92afba266cce4d1bad4134c351e74be3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:30:32.953898 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77a2796-8ccb-4ae6-9054-14973aa0c428 service nova] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:30:32.954182 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77a2796-8ccb-4ae6-9054-14973aa0c428 service nova] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:30:32.954456 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77a2796-8ccb-4ae6-9054-14973aa0c428 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:30:32.956140 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-d77a2796-8ccb-4ae6-9054-14973aa0c428 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:30:33.044614 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d77a2796-8ccb-4ae6-9054-14973aa0c428 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:30:33.046225 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d77a2796-8ccb-4ae6-9054-14973aa0c428 service nova] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:30:33.064498 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 704/1422] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jul 25 05:30:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 379 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 05:30:33.938415 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34a5d58-f5b0-48ad-97ea-44d7060de5aa None None] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:30:33.938971 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34a5d58-f5b0-48ad-97ea-44d7060de5aa None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:30:33.938971 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34a5d58-f5b0-48ad-97ea-44d7060de5aa None None] PATH_INFO: `/` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:30:33.940349 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 719/1423] 127.0.0.1 () {54 vars in 901 bytes} [Mon Jul 25 05:30:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:30:33.945968 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f59257-1fa7-4314-8ccf-a5d4b1e889e8 None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:30:33.946432 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f59257-1fa7-4314-8ccf-a5d4b1e889e8 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:30:33.946650 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f59257-1fa7-4314-8ccf-a5d4b1e889e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:30:33.966656 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:30:34.023444 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-09f59257-1fa7-4314-8ccf-a5d4b1e889e8 None None] MFA Rules not processed for user `1eae540242d94c10b0ad33960a855f08`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:30:34.024668 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-09f59257-1fa7-4314-8ccf-a5d4b1e889e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:30:34.121849 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 705/1424] 127.0.0.1 () {58 vars in 998 bytes} [Mon Jul 25 05:30:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:30:34.274018 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b34a5d58-f5b0-48ad-97ea-44d7060de5aa None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:30:34.275550 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-b34a5d58-f5b0-48ad-97ea-44d7060de5aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:30:34.385808 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:30:34.421094 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b34a5d58-f5b0-48ad-97ea-44d7060de5aa None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:30:34.533192 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1714f35a-0a6b-4c15-8b87-86429737ae55 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cd482a61e8a420db45bb532e5dae6e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92afba266cce4d1bad4134c351e74be3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:30:34.533847 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1714f35a-0a6b-4c15-8b87-86429737ae55 service nova] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:30:34.534141 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1714f35a-0a6b-4c15-8b87-86429737ae55 service nova] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:30:34.534980 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1714f35a-0a6b-4c15-8b87-86429737ae55 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:30:34.536608 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-1714f35a-0a6b-4c15-8b87-86429737ae55 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:30:34.614564 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:30:34.628167 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1714f35a-0a6b-4c15-8b87-86429737ae55 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:30:34.630248 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1714f35a-0a6b-4c15-8b87-86429737ae55 service nova] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:30:34.648559 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 720/1425] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jul 25 05:30:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 375 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 05:30:35.510974 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0dbfb79-d222-4e29-824e-a9613185d77c None None] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:30:35.511555 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0dbfb79-d222-4e29-824e-a9613185d77c None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:30:35.511555 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0dbfb79-d222-4e29-824e-a9613185d77c None None] PATH_INFO: `/` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:30:35.512945 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 706/1426] 127.0.0.1 () {54 vars in 901 bytes} [Mon Jul 25 05:30:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:30:35.517975 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5c4e8b-1f6f-4c03-85dc-d124def951c0 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:30:35.518429 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5c4e8b-1f6f-4c03-85dc-d124def951c0 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:30:35.518530 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5c4e8b-1f6f-4c03-85dc-d124def951c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:30:35.593616 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-ac5c4e8b-1f6f-4c03-85dc-d124def951c0 None None] MFA Rules not processed for user `1eae540242d94c10b0ad33960a855f08`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:30:35.594822 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-ac5c4e8b-1f6f-4c03-85dc-d124def951c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:30:35.694912 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 721/1427] 127.0.0.1 () {58 vars in 998 bytes} [Mon Jul 25 05:30:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:30:35.850275 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0dbfb79-d222-4e29-824e-a9613185d77c None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:30:35.851823 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-c0dbfb79-d222-4e29-824e-a9613185d77c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:30:35.939619 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:30:36.012224 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0dbfb79-d222-4e29-824e-a9613185d77c None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:30:36.135281 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb1c0c0-b69b-40cb-8417-dc4e84d7726e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cd482a61e8a420db45bb532e5dae6e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92afba266cce4d1bad4134c351e74be3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:30:36.135857 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb1c0c0-b69b-40cb-8417-dc4e84d7726e service nova] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:30:36.136144 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb1c0c0-b69b-40cb-8417-dc4e84d7726e service nova] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:30:36.136421 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb1c0c0-b69b-40cb-8417-dc4e84d7726e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:30:36.138141 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-0eb1c0c0-b69b-40cb-8417-dc4e84d7726e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:30:36.188687 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:30:36.231345 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eb1c0c0-b69b-40cb-8417-dc4e84d7726e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:30:36.232959 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eb1c0c0-b69b-40cb-8417-dc4e84d7726e service nova] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:30:36.251977 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 707/1428] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jul 25 05:30:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 402 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 05:30:37.073748 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2f56f5-84f0-4d43-82d9-b9d0987d276f None None] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:30:37.074275 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2f56f5-84f0-4d43-82d9-b9d0987d276f None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:30:37.074275 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2f56f5-84f0-4d43-82d9-b9d0987d276f None None] PATH_INFO: `/` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:30:37.075624 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 722/1429] 127.0.0.1 () {54 vars in 901 bytes} [Mon Jul 25 05:30:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:30:37.080702 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0e1147-0a9a-4300-bbb3-f42b95ae5a38 None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:30:37.081161 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0e1147-0a9a-4300-bbb3-f42b95ae5a38 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:30:37.081857 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0e1147-0a9a-4300-bbb3-f42b95ae5a38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:30:37.160615 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-2b0e1147-0a9a-4300-bbb3-f42b95ae5a38 None None] MFA Rules not processed for user `1eae540242d94c10b0ad33960a855f08`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:30:37.161868 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-2b0e1147-0a9a-4300-bbb3-f42b95ae5a38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:30:37.260579 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 708/1430] 127.0.0.1 () {58 vars in 998 bytes} [Mon Jul 25 05:30:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:30:37.411185 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe2f56f5-84f0-4d43-82d9-b9d0987d276f None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:30:37.412623 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-fe2f56f5-84f0-4d43-82d9-b9d0987d276f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:30:37.463607 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:30:37.563252 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe2f56f5-84f0-4d43-82d9-b9d0987d276f None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:30:37.682374 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45fdd2a3-e2a8-4ea6-ba3d-6e6656c6c633 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cd482a61e8a420db45bb532e5dae6e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92afba266cce4d1bad4134c351e74be3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:30:37.683093 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45fdd2a3-e2a8-4ea6-ba3d-6e6656c6c633 service nova] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:30:37.683393 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45fdd2a3-e2a8-4ea6-ba3d-6e6656c6c633 service nova] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:30:37.683703 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45fdd2a3-e2a8-4ea6-ba3d-6e6656c6c633 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:30:37.685548 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-45fdd2a3-e2a8-4ea6-ba3d-6e6656c6c633 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:30:37.694153 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:30:37.782896 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45fdd2a3-e2a8-4ea6-ba3d-6e6656c6c633 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:30:37.785245 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45fdd2a3-e2a8-4ea6-ba3d-6e6656c6c633 service nova] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:30:37.803874 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 723/1431] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jul 25 05:30:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 393 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 05:30:38.635007 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b684904-8025-40c9-82af-1e624ec94dfc None None] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:30:38.635649 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b684904-8025-40c9-82af-1e624ec94dfc None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:30:38.635649 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b684904-8025-40c9-82af-1e624ec94dfc None None] PATH_INFO: `/` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:30:38.637050 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 709/1432] 127.0.0.1 () {54 vars in 901 bytes} [Mon Jul 25 05:30:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:30:38.641984 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f9f1f9-8c49-472a-9174-d13397450b1a None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:30:38.642440 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f9f1f9-8c49-472a-9174-d13397450b1a None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:30:38.642572 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f9f1f9-8c49-472a-9174-d13397450b1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:30:38.727529 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-c7f9f1f9-8c49-472a-9174-d13397450b1a None None] MFA Rules not processed for user `1eae540242d94c10b0ad33960a855f08`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:30:38.728810 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-c7f9f1f9-8c49-472a-9174-d13397450b1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:30:38.800160 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:30:38.833198 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 724/1433] 127.0.0.1 () {58 vars in 998 bytes} [Mon Jul 25 05:30:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:30:38.987255 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b684904-8025-40c9-82af-1e624ec94dfc None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:30:38.988794 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-9b684904-8025-40c9-82af-1e624ec94dfc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:30:39.007968 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:30:39.141959 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b684904-8025-40c9-82af-1e624ec94dfc None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:30:39.233849 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:30:39.263795 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be47a35-a4bd-4b95-8c37-b25ead994158 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cd482a61e8a420db45bb532e5dae6e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92afba266cce4d1bad4134c351e74be3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:30:39.264586 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be47a35-a4bd-4b95-8c37-b25ead994158 service nova] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:30:39.264911 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be47a35-a4bd-4b95-8c37-b25ead994158 service nova] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:30:39.265194 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be47a35-a4bd-4b95-8c37-b25ead994158 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:30:39.266900 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-1be47a35-a4bd-4b95-8c37-b25ead994158 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:30:39.365617 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1be47a35-a4bd-4b95-8c37-b25ead994158 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:30:39.367227 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1be47a35-a4bd-4b95-8c37-b25ead994158 service nova] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:30:39.385842 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 710/1434] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jul 25 05:30:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 399 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 05:30:40.214881 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed591d4-30b0-4f01-a6d2-97a849cb8172 None None] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:30:40.215411 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed591d4-30b0-4f01-a6d2-97a849cb8172 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:30:40.215411 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed591d4-30b0-4f01-a6d2-97a849cb8172 None None] PATH_INFO: `/` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:30:40.216803 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 725/1435] 127.0.0.1 () {54 vars in 901 bytes} [Mon Jul 25 05:30:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:30:40.222110 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63479469-7610-4dcf-a06e-b4432e672965 None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:30:40.222624 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63479469-7610-4dcf-a06e-b4432e672965 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:30:40.222733 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63479469-7610-4dcf-a06e-b4432e672965 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:30:40.301216 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-63479469-7610-4dcf-a06e-b4432e672965 None None] MFA Rules not processed for user `1eae540242d94c10b0ad33960a855f08`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:30:40.302684 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-63479469-7610-4dcf-a06e-b4432e672965 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:30:40.346706 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:30:40.406111 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 711/1436] 127.0.0.1 () {58 vars in 998 bytes} [Mon Jul 25 05:30:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:30:40.560158 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed591d4-30b0-4f01-a6d2-97a849cb8172 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:30:40.561797 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-0ed591d4-30b0-4f01-a6d2-97a849cb8172 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:30:40.714474 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed591d4-30b0-4f01-a6d2-97a849cb8172 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:30:40.771851 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:30:40.833619 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf0af4d-4b55-4ab6-b4b2-5746d14acda5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cd482a61e8a420db45bb532e5dae6e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92afba266cce4d1bad4134c351e74be3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:30:40.834573 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf0af4d-4b55-4ab6-b4b2-5746d14acda5 service nova] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:30:40.834862 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf0af4d-4b55-4ab6-b4b2-5746d14acda5 service nova] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:30:40.835146 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf0af4d-4b55-4ab6-b4b2-5746d14acda5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:30:40.836847 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-1bf0af4d-4b55-4ab6-b4b2-5746d14acda5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:30:40.930786 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bf0af4d-4b55-4ab6-b4b2-5746d14acda5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:30:40.932399 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bf0af4d-4b55-4ab6-b4b2-5746d14acda5 service nova] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:30:40.951812 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 726/1437] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jul 25 05:30:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 392 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 05:30:41.778333 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97c4e2e-354c-4f94-b9fb-e94c7cd8dfab None None] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:30:41.778891 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97c4e2e-354c-4f94-b9fb-e94c7cd8dfab None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:30:41.778891 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97c4e2e-354c-4f94-b9fb-e94c7cd8dfab None None] PATH_INFO: `/` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:30:41.780300 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 712/1438] 127.0.0.1 () {54 vars in 901 bytes} [Mon Jul 25 05:30:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:30:41.785872 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f416f0-1c42-42c5-8a93-c2191d37c641 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:30:41.786317 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f416f0-1c42-42c5-8a93-c2191d37c641 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:30:41.786441 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f416f0-1c42-42c5-8a93-c2191d37c641 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:30:41.869093 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-c1f416f0-1c42-42c5-8a93-c2191d37c641 None None] MFA Rules not processed for user `1eae540242d94c10b0ad33960a855f08`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:30:41.870318 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-c1f416f0-1c42-42c5-8a93-c2191d37c641 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:30:41.872467 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:30:41.973585 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 727/1439] 127.0.0.1 () {58 vars in 998 bytes} [Mon Jul 25 05:30:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:30:42.126504 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a97c4e2e-354c-4f94-b9fb-e94c7cd8dfab None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:30:42.128056 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-a97c4e2e-354c-4f94-b9fb-e94c7cd8dfab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:30:42.282477 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a97c4e2e-354c-4f94-b9fb-e94c7cd8dfab None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:30:42.311008 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:30:42.399699 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7198f97d-1b3a-4aad-8f21-8ecb383602b9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cd482a61e8a420db45bb532e5dae6e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92afba266cce4d1bad4134c351e74be3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:30:42.400347 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7198f97d-1b3a-4aad-8f21-8ecb383602b9 service nova] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:30:42.400631 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7198f97d-1b3a-4aad-8f21-8ecb383602b9 service nova] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:30:42.400938 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7198f97d-1b3a-4aad-8f21-8ecb383602b9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:30:42.402593 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-7198f97d-1b3a-4aad-8f21-8ecb383602b9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:30:42.497705 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7198f97d-1b3a-4aad-8f21-8ecb383602b9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:30:42.499359 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7198f97d-1b3a-4aad-8f21-8ecb383602b9 service nova] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:30:42.518091 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 713/1440] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jul 25 05:30:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 392 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 05:30:44.476664 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86a3c14-f8b5-4e32-9c28-3a3f27c2826a None None] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:30:44.477222 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86a3c14-f8b5-4e32-9c28-3a3f27c2826a None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:30:44.477222 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86a3c14-f8b5-4e32-9c28-3a3f27c2826a None None] PATH_INFO: `/` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:30:44.478545 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 728/1441] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jul 25 05:30:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:30:44.483725 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa05ceaa-f4e0-45fd-8fd5-df2c8f7002d7 None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:30:44.484188 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa05ceaa-f4e0-45fd-8fd5-df2c8f7002d7 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:30:44.484287 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa05ceaa-f4e0-45fd-8fd5-df2c8f7002d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:30:44.522954 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:30:44.548419 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-aa05ceaa-f4e0-45fd-8fd5-df2c8f7002d7 None None] MFA Rules not processed for user `1eae540242d94c10b0ad33960a855f08`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:30:44.549655 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-aa05ceaa-f4e0-45fd-8fd5-df2c8f7002d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:30:44.653023 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 714/1442] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jul 25 05:30:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:30:44.694021 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d86a3c14-f8b5-4e32-9c28-3a3f27c2826a None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:30:44.695654 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-d86a3c14-f8b5-4e32-9c28-3a3f27c2826a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:30:44.805695 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d86a3c14-f8b5-4e32-9c28-3a3f27c2826a None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:30:44.839794 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:30:44.882232 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b2c101-59cf-4168-a6df-620ff8df3be1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b2931200f99e43d79a50a9f6e02738e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92afba266cce4d1bad4134c351e74be3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:30:44.883267 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b2c101-59cf-4168-a6df-620ff8df3be1 service neutron] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:30:44.883556 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b2c101-59cf-4168-a6df-620ff8df3be1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:30:44.883837 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b2c101-59cf-4168-a6df-620ff8df3be1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:30:44.885604 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-b2b2c101-59cf-4168-a6df-620ff8df3be1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:30:44.995671 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2b2c101-59cf-4168-a6df-620ff8df3be1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:30:44.997155 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2b2c101-59cf-4168-a6df-620ff8df3be1 service neutron] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:30:45.015783 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 729/1443] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jul 25 05:30:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 05:30:47.249796 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f2c691-13ee-46ee-97c5-1ab2c8e2e06d None None] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:30:47.250383 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f2c691-13ee-46ee-97c5-1ab2c8e2e06d None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:30:47.250383 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f2c691-13ee-46ee-97c5-1ab2c8e2e06d None None] PATH_INFO: `/` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:30:47.251760 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 715/1444] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jul 25 05:30:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:30:47.257703 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6480909-e184-4bd9-ad33-a28e858e62f5 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:30:47.258123 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6480909-e184-4bd9-ad33-a28e858e62f5 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:30:47.258269 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6480909-e184-4bd9-ad33-a28e858e62f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:30:47.321802 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-e6480909-e184-4bd9-ad33-a28e858e62f5 None None] MFA Rules not processed for user `1eae540242d94c10b0ad33960a855f08`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:30:47.323008 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-e6480909-e184-4bd9-ad33-a28e858e62f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:30:47.336486 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:30:47.536924 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 730/1445] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jul 25 05:30:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 280 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:30:47.575379 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f2c691-13ee-46ee-97c5-1ab2c8e2e06d None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:30:47.576959 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-09f2c691-13ee-46ee-97c5-1ab2c8e2e06d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:30:47.681648 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f2c691-13ee-46ee-97c5-1ab2c8e2e06d None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:30:47.694703 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:30:47.757631 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-396383c2-16a3-4649-86b0-6529c6d0294c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b2931200f99e43d79a50a9f6e02738e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92afba266cce4d1bad4134c351e74be3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:30:47.758282 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396383c2-16a3-4649-86b0-6529c6d0294c service neutron] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:30:47.758566 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396383c2-16a3-4649-86b0-6529c6d0294c service neutron] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:30:47.758844 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396383c2-16a3-4649-86b0-6529c6d0294c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:30:47.760495 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-396383c2-16a3-4649-86b0-6529c6d0294c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:30:47.870508 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-396383c2-16a3-4649-86b0-6529c6d0294c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:30:47.871940 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-396383c2-16a3-4649-86b0-6529c6d0294c service neutron] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:30:47.889776 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 716/1446] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jul 25 05:30:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 05:30:50.247973 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073064c2-30b1-40e8-ac3d-e78a76f9d9b0 None None] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:30:50.248507 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073064c2-30b1-40e8-ac3d-e78a76f9d9b0 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:30:50.248507 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073064c2-30b1-40e8-ac3d-e78a76f9d9b0 None None] PATH_INFO: `/` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:30:50.249861 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 731/1447] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jul 25 05:30:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:30:50.256241 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fcbdb68-fecd-4e69-846a-2511921236a7 None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:30:50.256690 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fcbdb68-fecd-4e69-846a-2511921236a7 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:30:50.256836 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fcbdb68-fecd-4e69-846a-2511921236a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:30:50.303929 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:30:50.322508 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-9fcbdb68-fecd-4e69-846a-2511921236a7 None None] MFA Rules not processed for user `1eae540242d94c10b0ad33960a855f08`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:30:50.323724 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-9fcbdb68-fecd-4e69-846a-2511921236a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:30:50.418591 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 717/1448] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jul 25 05:30:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:30:50.458635 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-073064c2-30b1-40e8-ac3d-e78a76f9d9b0 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:30:50.460399 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-073064c2-30b1-40e8-ac3d-e78a76f9d9b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:30:50.562032 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-073064c2-30b1-40e8-ac3d-e78a76f9d9b0 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:30:50.601996 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:30:50.636927 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a59f727d-2b8c-4847-98db-786be0a8b6aa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b2931200f99e43d79a50a9f6e02738e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92afba266cce4d1bad4134c351e74be3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:30:50.637592 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59f727d-2b8c-4847-98db-786be0a8b6aa service neutron] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:30:50.637875 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59f727d-2b8c-4847-98db-786be0a8b6aa service neutron] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:30:50.638150 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59f727d-2b8c-4847-98db-786be0a8b6aa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:30:50.639821 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-a59f727d-2b8c-4847-98db-786be0a8b6aa service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:30:50.746505 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a59f727d-2b8c-4847-98db-786be0a8b6aa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:30:50.747921 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a59f727d-2b8c-4847-98db-786be0a8b6aa service neutron] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:30:50.765901 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 732/1449] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jul 25 05:30:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 05:30:52.968934 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07258f45-fcce-4242-b200-e7bf1ae46e2f None None] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:30:52.969476 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07258f45-fcce-4242-b200-e7bf1ae46e2f None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:30:52.969476 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07258f45-fcce-4242-b200-e7bf1ae46e2f None None] PATH_INFO: `/` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:30:52.970944 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 718/1450] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jul 25 05:30:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:30:52.976546 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eaa79bc-5a72-4cad-9b44-98da83fae1bc None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:30:52.976964 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eaa79bc-5a72-4cad-9b44-98da83fae1bc None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:30:52.977155 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eaa79bc-5a72-4cad-9b44-98da83fae1bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:30:53.041733 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-7eaa79bc-5a72-4cad-9b44-98da83fae1bc None None] MFA Rules not processed for user `1eae540242d94c10b0ad33960a855f08`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:30:53.042946 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-7eaa79bc-5a72-4cad-9b44-98da83fae1bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:30:53.063655 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:30:53.141051 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 733/1451] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jul 25 05:30:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:30:53.180863 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07258f45-fcce-4242-b200-e7bf1ae46e2f None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:30:53.182388 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-07258f45-fcce-4242-b200-e7bf1ae46e2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:30:53.284547 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07258f45-fcce-4242-b200-e7bf1ae46e2f None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:30:53.305117 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:30:53.358123 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db98b25b-ef20-430c-9cf1-f8cf68885350 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b2931200f99e43d79a50a9f6e02738e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92afba266cce4d1bad4134c351e74be3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:30:53.358687 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db98b25b-ef20-430c-9cf1-f8cf68885350 service neutron] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:30:53.358971 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db98b25b-ef20-430c-9cf1-f8cf68885350 service neutron] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:30:53.359245 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db98b25b-ef20-430c-9cf1-f8cf68885350 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:30:53.360891 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-db98b25b-ef20-430c-9cf1-f8cf68885350 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:30:53.467726 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db98b25b-ef20-430c-9cf1-f8cf68885350 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:30:53.469179 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db98b25b-ef20-430c-9cf1-f8cf68885350 service neutron] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:30:53.487217 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 719/1452] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jul 25 05:30:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 05:30:55.585558 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb603d6-e197-4fa1-a104-0c1a6996f97d None None] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:30:55.586121 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb603d6-e197-4fa1-a104-0c1a6996f97d None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:30:55.586121 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb603d6-e197-4fa1-a104-0c1a6996f97d None None] PATH_INFO: `/` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:30:55.587514 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 734/1453] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jul 25 05:30:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:30:55.592732 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3348e3-070b-47fe-9896-cc04f7ffc72a None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:30:55.593166 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3348e3-070b-47fe-9896-cc04f7ffc72a None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:30:55.593337 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3348e3-070b-47fe-9896-cc04f7ffc72a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:30:55.654866 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:30:55.657572 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-8a3348e3-070b-47fe-9896-cc04f7ffc72a None None] MFA Rules not processed for user `1eae540242d94c10b0ad33960a855f08`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:30:55.659364 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-8a3348e3-070b-47fe-9896-cc04f7ffc72a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:30:55.754922 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 720/1454] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jul 25 05:30:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:30:55.794152 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb603d6-e197-4fa1-a104-0c1a6996f97d None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:30:55.795709 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-fcb603d6-e197-4fa1-a104-0c1a6996f97d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:30:55.898954 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb603d6-e197-4fa1-a104-0c1a6996f97d None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:30:55.947791 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:30:55.971984 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d525512d-3575-4798-83c9-812e3eb5d0b7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b2931200f99e43d79a50a9f6e02738e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92afba266cce4d1bad4134c351e74be3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:30:55.972629 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d525512d-3575-4798-83c9-812e3eb5d0b7 service neutron] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:30:55.973103 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d525512d-3575-4798-83c9-812e3eb5d0b7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:30:55.973550 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d525512d-3575-4798-83c9-812e3eb5d0b7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:30:55.975193 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-d525512d-3575-4798-83c9-812e3eb5d0b7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:30:56.083549 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d525512d-3575-4798-83c9-812e3eb5d0b7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:30:56.085012 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d525512d-3575-4798-83c9-812e3eb5d0b7 service neutron] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:30:56.104492 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 735/1455] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jul 25 05:30:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 05:30:58.200773 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc7359f-b12f-4de5-80a5-e916f03c1549 None None] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:30:58.201353 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc7359f-b12f-4de5-80a5-e916f03c1549 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:30:58.201353 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc7359f-b12f-4de5-80a5-e916f03c1549 None None] PATH_INFO: `/` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:30:58.202697 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 721/1456] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jul 25 05:30:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:30:58.208755 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed12e75-5867-4526-bd58-f1107dc5d93b None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:30:58.209231 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed12e75-5867-4526-bd58-f1107dc5d93b None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:30:58.209345 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed12e75-5867-4526-bd58-f1107dc5d93b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:30:58.272818 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-aed12e75-5867-4526-bd58-f1107dc5d93b None None] MFA Rules not processed for user `1eae540242d94c10b0ad33960a855f08`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:30:58.274033 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-aed12e75-5867-4526-bd58-f1107dc5d93b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:30:58.303568 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:30:58.371172 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 736/1457] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jul 25 05:30:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:30:58.409454 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc7359f-b12f-4de5-80a5-e916f03c1549 None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:30:58.411691 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-3dc7359f-b12f-4de5-80a5-e916f03c1549 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:30:58.514162 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc7359f-b12f-4de5-80a5-e916f03c1549 None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:30:58.542494 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:30:58.587037 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b2da67-93b5-45fe-8a3f-036d5f7237c0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b2931200f99e43d79a50a9f6e02738e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92afba266cce4d1bad4134c351e74be3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:30:58.587693 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b2da67-93b5-45fe-8a3f-036d5f7237c0 service neutron] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:30:58.587977 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b2da67-93b5-45fe-8a3f-036d5f7237c0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:30:58.588419 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b2da67-93b5-45fe-8a3f-036d5f7237c0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:30:58.590503 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-e8b2da67-93b5-45fe-8a3f-036d5f7237c0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:30:58.697620 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8b2da67-93b5-45fe-8a3f-036d5f7237c0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:30:58.699065 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8b2da67-93b5-45fe-8a3f-036d5f7237c0 service neutron] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:30:58.717998 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 722/1458] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jul 25 05:30:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 05:31:00.796194 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64de372b-7e80-4eb3-be34-7df58e6a7051 None None] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:31:00.796913 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64de372b-7e80-4eb3-be34-7df58e6a7051 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:31:00.796913 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64de372b-7e80-4eb3-be34-7df58e6a7051 None None] PATH_INFO: `/` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:31:00.798233 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 737/1459] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jul 25 05:31:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:31:00.804657 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d57a3a-ac9f-4d4e-8782-5fb3659aa018 None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:31:00.805138 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d57a3a-ac9f-4d4e-8782-5fb3659aa018 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:31:00.805264 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d57a3a-ac9f-4d4e-8782-5fb3659aa018 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:31:00.871445 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-a5d57a3a-ac9f-4d4e-8782-5fb3659aa018 None None] MFA Rules not processed for user `1eae540242d94c10b0ad33960a855f08`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:31:00.872645 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-a5d57a3a-ac9f-4d4e-8782-5fb3659aa018 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:31:00.881176 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:31:00.969694 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 723/1460] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jul 25 05:31:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:31:01.008602 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64de372b-7e80-4eb3-be34-7df58e6a7051 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:31:01.010150 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-64de372b-7e80-4eb3-be34-7df58e6a7051 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:31:01.112415 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64de372b-7e80-4eb3-be34-7df58e6a7051 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:31:01.167308 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:31:01.185618 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-909b5c3f-875f-4a46-8e8c-f89349d23aca service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b2931200f99e43d79a50a9f6e02738e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92afba266cce4d1bad4134c351e74be3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:31:01.186302 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909b5c3f-875f-4a46-8e8c-f89349d23aca service neutron] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:31:01.186587 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909b5c3f-875f-4a46-8e8c-f89349d23aca service neutron] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:31:01.186865 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909b5c3f-875f-4a46-8e8c-f89349d23aca service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:31:01.188516 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-909b5c3f-875f-4a46-8e8c-f89349d23aca service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:31:01.295115 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-909b5c3f-875f-4a46-8e8c-f89349d23aca service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:31:01.296540 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-909b5c3f-875f-4a46-8e8c-f89349d23aca service neutron] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:31:01.314942 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 738/1461] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jul 25 05:31:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 05:31:03.390693 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d25e7e9-b5c6-43ed-89fc-81075ae2a454 None None] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:31:03.391243 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d25e7e9-b5c6-43ed-89fc-81075ae2a454 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:31:03.391243 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d25e7e9-b5c6-43ed-89fc-81075ae2a454 None None] PATH_INFO: `/` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:31:03.392623 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 724/1462] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jul 25 05:31:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:31:03.398634 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786f60ec-2a41-42a0-9e3f-db134a603dd1 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:31:03.399069 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786f60ec-2a41-42a0-9e3f-db134a603dd1 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:31:03.399220 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786f60ec-2a41-42a0-9e3f-db134a603dd1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:31:03.463150 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-786f60ec-2a41-42a0-9e3f-db134a603dd1 None None] MFA Rules not processed for user `1eae540242d94c10b0ad33960a855f08`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:31:03.464349 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-786f60ec-2a41-42a0-9e3f-db134a603dd1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:31:03.501881 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:31:03.559610 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 739/1463] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jul 25 05:31:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:31:03.600010 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d25e7e9-b5c6-43ed-89fc-81075ae2a454 None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:31:03.601580 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-6d25e7e9-b5c6-43ed-89fc-81075ae2a454 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:31:03.703612 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d25e7e9-b5c6-43ed-89fc-81075ae2a454 None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:31:03.740977 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:31:03.776733 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a96b00a-51a1-4d2f-879c-6f8e5842eff8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b2931200f99e43d79a50a9f6e02738e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92afba266cce4d1bad4134c351e74be3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:31:03.777402 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a96b00a-51a1-4d2f-879c-6f8e5842eff8 service neutron] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:31:03.777685 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a96b00a-51a1-4d2f-879c-6f8e5842eff8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:31:03.777959 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a96b00a-51a1-4d2f-879c-6f8e5842eff8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:31:03.779587 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-2a96b00a-51a1-4d2f-879c-6f8e5842eff8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:31:03.892166 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a96b00a-51a1-4d2f-879c-6f8e5842eff8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:31:03.893671 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a96b00a-51a1-4d2f-879c-6f8e5842eff8 service neutron] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:31:03.912176 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 725/1464] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jul 25 05:31:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 05:31:06.024809 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf615fc-94d7-436a-89dc-a7788bd75410 None None] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:31:06.025381 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf615fc-94d7-436a-89dc-a7788bd75410 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:31:06.025500 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf615fc-94d7-436a-89dc-a7788bd75410 None None] PATH_INFO: `/` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:31:06.026989 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 740/1465] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jul 25 05:31:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:31:06.032431 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f85e601-3362-4202-a988-e89a352669a3 None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:31:06.032896 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f85e601-3362-4202-a988-e89a352669a3 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:31:06.033007 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f85e601-3362-4202-a988-e89a352669a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:31:06.102621 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-6f85e601-3362-4202-a988-e89a352669a3 None None] MFA Rules not processed for user `1eae540242d94c10b0ad33960a855f08`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:31:06.103754 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-6f85e601-3362-4202-a988-e89a352669a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:31:06.122595 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:31:06.202057 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 726/1466] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jul 25 05:31:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:31:06.241011 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf615fc-94d7-436a-89dc-a7788bd75410 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:31:06.242546 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-bcf615fc-94d7-436a-89dc-a7788bd75410 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:31:06.366775 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf615fc-94d7-436a-89dc-a7788bd75410 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:31:06.432414 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:31:06.439677 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f68793-f56e-484f-8207-fb4df6af41a3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b2931200f99e43d79a50a9f6e02738e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92afba266cce4d1bad4134c351e74be3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:31:06.440332 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f68793-f56e-484f-8207-fb4df6af41a3 service neutron] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:31:06.440622 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f68793-f56e-484f-8207-fb4df6af41a3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:31:06.440939 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f68793-f56e-484f-8207-fb4df6af41a3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:31:06.442779 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-a3f68793-f56e-484f-8207-fb4df6af41a3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:31:06.549826 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3f68793-f56e-484f-8207-fb4df6af41a3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:31:06.551299 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3f68793-f56e-484f-8207-fb4df6af41a3 service neutron] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:31:06.569248 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 741/1467] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jul 25 05:31:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 05:31:08.657622 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a12574-0c52-4056-988d-2fc24a4295d4 None None] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:31:08.658160 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a12574-0c52-4056-988d-2fc24a4295d4 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:31:08.658160 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a12574-0c52-4056-988d-2fc24a4295d4 None None] PATH_INFO: `/` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:31:08.659494 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 727/1468] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jul 25 05:31:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:31:08.665459 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0cc184-7a2c-4f64-896d-dd07f4398fd0 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:31:08.665915 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0cc184-7a2c-4f64-896d-dd07f4398fd0 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:31:08.666026 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0cc184-7a2c-4f64-896d-dd07f4398fd0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:31:08.732075 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-6e0cc184-7a2c-4f64-896d-dd07f4398fd0 None None] MFA Rules not processed for user `1eae540242d94c10b0ad33960a855f08`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:31:08.733333 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-6e0cc184-7a2c-4f64-896d-dd07f4398fd0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:31:08.781224 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:31:08.836119 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 742/1469] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jul 25 05:31:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:31:08.873842 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a12574-0c52-4056-988d-2fc24a4295d4 None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:31:08.875382 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-b5a12574-0c52-4056-988d-2fc24a4295d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:31:08.979221 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a12574-0c52-4056-988d-2fc24a4295d4 None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:31:09.027476 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:31:09.053488 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c29321c9-a52f-456a-a00f-b741b2e95cdd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b2931200f99e43d79a50a9f6e02738e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92afba266cce4d1bad4134c351e74be3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:31:09.054137 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29321c9-a52f-456a-a00f-b741b2e95cdd service neutron] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:31:09.054421 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29321c9-a52f-456a-a00f-b741b2e95cdd service neutron] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:31:09.054699 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29321c9-a52f-456a-a00f-b741b2e95cdd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:31:09.056339 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-c29321c9-a52f-456a-a00f-b741b2e95cdd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:31:09.163893 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c29321c9-a52f-456a-a00f-b741b2e95cdd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:31:09.165355 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c29321c9-a52f-456a-a00f-b741b2e95cdd service neutron] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:31:09.182945 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 728/1470] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jul 25 05:31:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 05:31:18.834051 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=22 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:31:19.180671 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=66 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:31:28.923533 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf4edef-d92d-4051-be65-5a87470fa3d5 None None] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:31:28.924188 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf4edef-d92d-4051-be65-5a87470fa3d5 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:31:28.924188 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf4edef-d92d-4051-be65-5a87470fa3d5 None None] PATH_INFO: `/` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:31:28.925535 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 743/1471] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jul 25 05:31:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:31:28.930874 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ac6006-6874-4c7c-8d4e-743380c9468c None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:31:28.931331 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ac6006-6874-4c7c-8d4e-743380c9468c None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:31:28.931443 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ac6006-6874-4c7c-8d4e-743380c9468c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:31:28.996345 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-18ac6006-6874-4c7c-8d4e-743380c9468c None None] MFA Rules not processed for user `1eae540242d94c10b0ad33960a855f08`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:31:28.997646 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-18ac6006-6874-4c7c-8d4e-743380c9468c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:31:29.094815 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 729/1472] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jul 25 05:31:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:31:29.180195 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f892a238-b67d-47fb-808b-4ef50ba85d19 None None] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:31:29.180487 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f892a238-b67d-47fb-808b-4ef50ba85d19 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:31:29.180771 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f892a238-b67d-47fb-808b-4ef50ba85d19 None None] PATH_INFO: `/` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:31:29.182173 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 744/1473] 10.222.0.33 () {54 vars in 966 bytes} [Mon Jul 25 05:31:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:31:29.186994 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414764b8-f260-4eb5-8c18-75ca9766cc94 None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:31:29.187289 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414764b8-f260-4eb5-8c18-75ca9766cc94 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:31:29.187572 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414764b8-f260-4eb5-8c18-75ca9766cc94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:31:29.262671 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-414764b8-f260-4eb5-8c18-75ca9766cc94 None None] MFA Rules not processed for user `de99e44bdf9c4030adbe9c0efa3831cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:31:29.263888 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-414764b8-f260-4eb5-8c18-75ca9766cc94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:31:29.284701 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:31:29.339139 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 730/1474] 10.222.0.33 () {58 vars in 1063 bytes} [Mon Jul 25 05:31:29 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:31:29.346989 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f892a238-b67d-47fb-808b-4ef50ba85d19 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:31:29.348435 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-f892a238-b67d-47fb-808b-4ef50ba85d19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:31:29.449883 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f892a238-b67d-47fb-808b-4ef50ba85d19 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:31:29.524117 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e596c928-6360-4d4a-9bda-95b2c8d4d4cb service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de99e44bdf9c4030adbe9c0efa3831cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92afba266cce4d1bad4134c351e74be3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:31:29.524822 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e596c928-6360-4d4a-9bda-95b2c8d4d4cb service glance] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:31:29.525110 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e596c928-6360-4d4a-9bda-95b2c8d4d4cb service glance] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:31:29.525384 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e596c928-6360-4d4a-9bda-95b2c8d4d4cb service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:31:29.527023 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-e596c928-6360-4d4a-9bda-95b2c8d4d4cb service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:31:29.580677 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:31:29.634478 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e596c928-6360-4d4a-9bda-95b2c8d4d4cb service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:31:29.635834 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e596c928-6360-4d4a-9bda-95b2c8d4d4cb service glance] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:31:29.654374 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 745/1475] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jul 25 05:31:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 05:31:31.784139 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66a2886-3b6e-49d8-9075-ce95b9d1b406 None None] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:31:31.784721 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66a2886-3b6e-49d8-9075-ce95b9d1b406 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:31:31.784721 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66a2886-3b6e-49d8-9075-ce95b9d1b406 None None] PATH_INFO: `/` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:31:31.786124 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 731/1476] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jul 25 05:31:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 25 05:31:31.791385 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599ad9c9-5e8c-45f1-abdd-79fcc1609632 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:31:31.791807 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599ad9c9-5e8c-45f1-abdd-79fcc1609632 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:31:31.791971 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599ad9c9-5e8c-45f1-abdd-79fcc1609632 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:31:31.856986 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-599ad9c9-5e8c-45f1-abdd-79fcc1609632 None None] MFA Rules not processed for user `1eae540242d94c10b0ad33960a855f08`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:31:31.858185 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-599ad9c9-5e8c-45f1-abdd-79fcc1609632 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:31:31.950726 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 746/1477] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jul 25 05:31:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:31:31.975545 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b66a2886-3b6e-49d8-9075-ce95b9d1b406 None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:31:31.977101 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-b66a2886-3b6e-49d8-9075-ce95b9d1b406 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:31:32.102220 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b66a2886-3b6e-49d8-9075-ce95b9d1b406 None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:31:32.150061 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:31:32.195168 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17bdff16-74cb-4464-9067-9f81ca83608b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:31:32.195824 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17bdff16-74cb-4464-9067-9f81ca83608b admin admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:31:32.196114 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17bdff16-74cb-4464-9067-9f81ca83608b admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:31:32.196387 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17bdff16-74cb-4464-9067-9f81ca83608b admin admin] PATH_INFO: `/v3/projects` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:31:32.197152 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17bdff16-74cb-4464-9067-9f81ca83608b admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:31:32.198948 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: WARNING py.warnings [None req-17bdff16-74cb-4464-9067-9f81ca83608b admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 25 05:31:32.198948 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: warnings.warn(msg) Jul 25 05:31:32.198948 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: Jul 25 05:31:32.200077 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17bdff16-74cb-4464-9067-9f81ca83608b admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:31:32.219076 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 732/1478] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Jul 25 05:31:31 2022] GET /identity/v3/projects => generated 2553 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 05:31:34.306795 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4298ca57-e1d1-4dc7-bb20-b910a05755e6 None None] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:31:34.307349 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4298ca57-e1d1-4dc7-bb20-b910a05755e6 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:31:34.307349 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4298ca57-e1d1-4dc7-bb20-b910a05755e6 None None] PATH_INFO: `/` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:31:34.308765 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 747/1479] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jul 25 05:31:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:31:34.314086 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc540bbe-fe80-43ae-bc4e-cb19047cd32a None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:31:34.314504 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc540bbe-fe80-43ae-bc4e-cb19047cd32a None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:31:34.314654 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc540bbe-fe80-43ae-bc4e-cb19047cd32a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:31:34.379091 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-cc540bbe-fe80-43ae-bc4e-cb19047cd32a None None] MFA Rules not processed for user `1eae540242d94c10b0ad33960a855f08`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:31:34.380317 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-cc540bbe-fe80-43ae-bc4e-cb19047cd32a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:31:34.478894 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 733/1480] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jul 25 05:31:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:31:34.523111 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4298ca57-e1d1-4dc7-bb20-b910a05755e6 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:31:34.524642 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-4298ca57-e1d1-4dc7-bb20-b910a05755e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:31:34.534679 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:31:34.670740 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4298ca57-e1d1-4dc7-bb20-b910a05755e6 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:31:34.750598 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:31:34.783143 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e0c22b-b38d-4b40-af53-96724e6f2ceb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cd482a61e8a420db45bb532e5dae6e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92afba266cce4d1bad4134c351e74be3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:31:34.783855 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e0c22b-b38d-4b40-af53-96724e6f2ceb service nova] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:31:34.784141 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e0c22b-b38d-4b40-af53-96724e6f2ceb service nova] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:31:34.784417 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e0c22b-b38d-4b40-af53-96724e6f2ceb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:31:34.786079 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-04e0c22b-b38d-4b40-af53-96724e6f2ceb service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:31:34.874016 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04e0c22b-b38d-4b40-af53-96724e6f2ceb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:31:34.875598 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04e0c22b-b38d-4b40-af53-96724e6f2ceb service nova] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:31:34.893394 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 748/1481] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jul 25 05:31:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 371 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 05:31:37.119852 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd761124-902f-45ff-9547-1352465daedb None None] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:31:37.120416 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd761124-902f-45ff-9547-1352465daedb None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:31:37.120416 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd761124-902f-45ff-9547-1352465daedb None None] PATH_INFO: `/` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:31:37.121722 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 734/1482] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jul 25 05:31:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:31:37.127430 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e173f1-8034-4e9f-ac10-d85ae929cb1c None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:31:37.127867 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e173f1-8034-4e9f-ac10-d85ae929cb1c None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:31:37.127995 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e173f1-8034-4e9f-ac10-d85ae929cb1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:31:37.193185 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-c6e173f1-8034-4e9f-ac10-d85ae929cb1c None None] MFA Rules not processed for user `1eae540242d94c10b0ad33960a855f08`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:31:37.194420 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-c6e173f1-8034-4e9f-ac10-d85ae929cb1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:31:37.239391 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:31:37.290061 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 749/1483] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jul 25 05:31:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:31:37.327180 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd761124-902f-45ff-9547-1352465daedb None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:31:37.328718 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-bd761124-902f-45ff-9547-1352465daedb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:31:37.352423 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:31:37.431405 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd761124-902f-45ff-9547-1352465daedb None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:31:37.504559 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-893f9655-c5f4-43be-9375-006c9e2119e1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b2931200f99e43d79a50a9f6e02738e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92afba266cce4d1bad4134c351e74be3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:31:37.505242 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893f9655-c5f4-43be-9375-006c9e2119e1 service neutron] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:31:37.505530 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893f9655-c5f4-43be-9375-006c9e2119e1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:31:37.505812 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893f9655-c5f4-43be-9375-006c9e2119e1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:31:37.507449 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-893f9655-c5f4-43be-9375-006c9e2119e1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:31:37.580658 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:31:37.617003 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-893f9655-c5f4-43be-9375-006c9e2119e1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:31:37.618480 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-893f9655-c5f4-43be-9375-006c9e2119e1 service neutron] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:31:37.636735 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 735/1484] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jul 25 05:31:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 05:31:39.713727 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05842475-a338-4b9b-bf20-6eb970fe9d17 None None] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:31:39.714271 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05842475-a338-4b9b-bf20-6eb970fe9d17 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:31:39.714271 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05842475-a338-4b9b-bf20-6eb970fe9d17 None None] PATH_INFO: `/` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:31:39.715672 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 750/1485] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jul 25 05:31:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:31:39.721622 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3a9207-cd74-4e71-82bd-68045104bf83 None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:31:39.722068 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3a9207-cd74-4e71-82bd-68045104bf83 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:31:39.722183 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3a9207-cd74-4e71-82bd-68045104bf83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:31:39.786771 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-9c3a9207-cd74-4e71-82bd-68045104bf83 None None] MFA Rules not processed for user `1eae540242d94c10b0ad33960a855f08`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:31:39.788004 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-9c3a9207-cd74-4e71-82bd-68045104bf83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:31:39.884939 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 736/1486] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jul 25 05:31:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:31:39.924623 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05842475-a338-4b9b-bf20-6eb970fe9d17 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:31:39.926319 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-05842475-a338-4b9b-bf20-6eb970fe9d17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:31:40.032155 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05842475-a338-4b9b-bf20-6eb970fe9d17 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:31:40.106030 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:31:40.107797 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dfbb6f9-79ab-4ed5-ba01-b8eef3b92802 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b2931200f99e43d79a50a9f6e02738e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92afba266cce4d1bad4134c351e74be3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:31:40.108538 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dfbb6f9-79ab-4ed5-ba01-b8eef3b92802 service neutron] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:31:40.109046 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dfbb6f9-79ab-4ed5-ba01-b8eef3b92802 service neutron] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:31:40.109388 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dfbb6f9-79ab-4ed5-ba01-b8eef3b92802 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:31:40.111814 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-7dfbb6f9-79ab-4ed5-ba01-b8eef3b92802 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:31:40.218710 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dfbb6f9-79ab-4ed5-ba01-b8eef3b92802 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:31:40.220170 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dfbb6f9-79ab-4ed5-ba01-b8eef3b92802 service neutron] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:31:40.237902 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 751/1487] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jul 25 05:31:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 05:31:42.484710 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05435218-612b-4f81-90d2-bf9a7eb55610 None None] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:31:42.485354 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05435218-612b-4f81-90d2-bf9a7eb55610 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:31:42.485354 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05435218-612b-4f81-90d2-bf9a7eb55610 None None] PATH_INFO: `/` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:31:42.486856 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 737/1488] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jul 25 05:31:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:31:42.492181 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a00a5a5-e7eb-4deb-8903-ee6aacce7483 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:31:42.492642 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a00a5a5-e7eb-4deb-8903-ee6aacce7483 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:31:42.492739 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a00a5a5-e7eb-4deb-8903-ee6aacce7483 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:31:42.559009 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-4a00a5a5-e7eb-4deb-8903-ee6aacce7483 None None] MFA Rules not processed for user `1eae540242d94c10b0ad33960a855f08`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:31:42.560228 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-4a00a5a5-e7eb-4deb-8903-ee6aacce7483 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:31:42.615526 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:31:42.659646 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 752/1489] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jul 25 05:31:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:31:42.701541 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05435218-612b-4f81-90d2-bf9a7eb55610 None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:31:42.703089 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-05435218-612b-4f81-90d2-bf9a7eb55610 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:31:42.739420 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:31:42.807487 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05435218-612b-4f81-90d2-bf9a7eb55610 None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:31:42.879449 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3af50a-9453-4f78-a57b-ec1226b592cc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b2931200f99e43d79a50a9f6e02738e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92afba266cce4d1bad4134c351e74be3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:31:42.880017 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3af50a-9453-4f78-a57b-ec1226b592cc service neutron] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:31:42.880300 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3af50a-9453-4f78-a57b-ec1226b592cc service neutron] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:31:42.880576 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3af50a-9453-4f78-a57b-ec1226b592cc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:31:42.882249 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-5b3af50a-9453-4f78-a57b-ec1226b592cc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:31:42.959827 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:31:42.988900 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b3af50a-9453-4f78-a57b-ec1226b592cc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:31:42.990346 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b3af50a-9453-4f78-a57b-ec1226b592cc service neutron] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:31:43.008657 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 738/1490] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jul 25 05:31:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 05:31:45.263018 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4a957f-4eea-4fe2-bd97-1de20ceac655 None None] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:31:45.263602 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4a957f-4eea-4fe2-bd97-1de20ceac655 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:31:45.263602 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4a957f-4eea-4fe2-bd97-1de20ceac655 None None] PATH_INFO: `/` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:31:45.264991 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 753/1491] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jul 25 05:31:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:31:45.270756 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff837653-b84f-4d42-9f61-065cbadb1264 None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:31:45.271225 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff837653-b84f-4d42-9f61-065cbadb1264 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:31:45.271334 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff837653-b84f-4d42-9f61-065cbadb1264 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:31:45.335251 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-ff837653-b84f-4d42-9f61-065cbadb1264 None None] MFA Rules not processed for user `1eae540242d94c10b0ad33960a855f08`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:31:45.336446 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-ff837653-b84f-4d42-9f61-065cbadb1264 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:31:45.430170 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 739/1492] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jul 25 05:31:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:31:45.469860 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc4a957f-4eea-4fe2-bd97-1de20ceac655 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:31:45.471398 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-dc4a957f-4eea-4fe2-bd97-1de20ceac655 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:31:45.573584 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc4a957f-4eea-4fe2-bd97-1de20ceac655 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:31:45.647549 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad463cc-8e51-4bc3-8bc9-33ac5c21ca6a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b2931200f99e43d79a50a9f6e02738e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92afba266cce4d1bad4134c351e74be3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:31:45.648207 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad463cc-8e51-4bc3-8bc9-33ac5c21ca6a service neutron] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:31:45.648492 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad463cc-8e51-4bc3-8bc9-33ac5c21ca6a service neutron] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:31:45.648770 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad463cc-8e51-4bc3-8bc9-33ac5c21ca6a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:31:45.650449 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-fad463cc-8e51-4bc3-8bc9-33ac5c21ca6a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:31:45.659680 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:31:45.760631 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fad463cc-8e51-4bc3-8bc9-33ac5c21ca6a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:31:45.762092 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fad463cc-8e51-4bc3-8bc9-33ac5c21ca6a service neutron] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:31:45.779910 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 754/1493] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jul 25 05:31:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 05:31:55.428365 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=82 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:31:55.777672 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=50 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:33:19.450455 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5456d8d4-c2b1-40e0-a774-3ca0f4d2c681 None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:33:19.451068 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5456d8d4-c2b1-40e0-a774-3ca0f4d2c681 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:33:19.451068 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5456d8d4-c2b1-40e0-a774-3ca0f4d2c681 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:33:19.523676 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-5456d8d4-c2b1-40e0-a774-3ca0f4d2c681 None None] MFA Rules not processed for user `1eae540242d94c10b0ad33960a855f08`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:33:19.524903 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-5456d8d4-c2b1-40e0-a774-3ca0f4d2c681 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:33:19.739341 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 740/1494] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 25 05:33:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 290 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:33:19.748114 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b337b8dd-ded5-4614-93b1-0c3019083ea9 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:33:19.748616 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b337b8dd-ded5-4614-93b1-0c3019083ea9 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:33:19.748722 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b337b8dd-ded5-4614-93b1-0c3019083ea9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:33:19.823075 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-b337b8dd-ded5-4614-93b1-0c3019083ea9 None None] MFA Rules not processed for user `1eae540242d94c10b0ad33960a855f08`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:33:19.824265 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-b337b8dd-ded5-4614-93b1-0c3019083ea9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:33:19.918687 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 755/1495] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 25 05:33:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:33:19.923296 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5456d8d4-c2b1-40e0-a774-3ca0f4d2c681 None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:33:19.924486 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-5456d8d4-c2b1-40e0-a774-3ca0f4d2c681 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:33:19.935670 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5456d8d4-c2b1-40e0-a774-3ca0f4d2c681 None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:33:19.995064 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:33:20.028119 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72bb0045-95d1-4fe8-a55f-a9021a5c4e79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:33:20.028767 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72bb0045-95d1-4fe8-a55f-a9021a5c4e79 admin admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:33:20.029076 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72bb0045-95d1-4fe8-a55f-a9021a5c4e79 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:33:20.029350 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72bb0045-95d1-4fe8-a55f-a9021a5c4e79 admin admin] PATH_INFO: `/v3/domains` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:33:20.030066 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72bb0045-95d1-4fe8-a55f-a9021a5c4e79 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:33:20.030369 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72bb0045-95d1-4fe8-a55f-a9021a5c4e79 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:33:20.031377 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: WARNING py.warnings [None req-72bb0045-95d1-4fe8-a55f-a9021a5c4e79 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 25 05:33:20.031377 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: warnings.warn(msg) Jul 25 05:33:20.031377 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: Jul 25 05:33:20.032245 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72bb0045-95d1-4fe8-a55f-a9021a5c4e79 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:33:20.043846 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 741/1496] 10.222.0.33 () {58 vars in 1152 bytes} [Mon Jul 25 05:33:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 121 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:33:20.052387 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b337b8dd-ded5-4614-93b1-0c3019083ea9 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:33:20.061592 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b337b8dd-ded5-4614-93b1-0c3019083ea9 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:33:20.122339 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:33:20.156535 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2126496c-3c10-4d18-8a23-decacf6d2fbf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:33:20.157226 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2126496c-3c10-4d18-8a23-decacf6d2fbf admin admin] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:33:20.157512 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2126496c-3c10-4d18-8a23-decacf6d2fbf admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:33:20.157789 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2126496c-3c10-4d18-8a23-decacf6d2fbf admin admin] PATH_INFO: `/v3/projects` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:33:20.158489 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2126496c-3c10-4d18-8a23-decacf6d2fbf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:33:20.160225 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: WARNING py.warnings [None req-2126496c-3c10-4d18-8a23-decacf6d2fbf admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 25 05:33:20.160225 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: warnings.warn(msg) Jul 25 05:33:20.160225 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: Jul 25 05:33:20.161456 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2126496c-3c10-4d18-8a23-decacf6d2fbf admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:33:20.216297 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 756/1497] 10.222.0.33 () {60 vars in 1152 bytes} [Mon Jul 25 05:33:20 2022] POST /identity/v3/projects => generated 379 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 05:33:20.220448 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72bb0045-95d1-4fe8-a55f-a9021a5c4e79 admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:33:20.222028 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-72bb0045-95d1-4fe8-a55f-a9021a5c4e79 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:33:20.254684 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72bb0045-95d1-4fe8-a55f-a9021a5c4e79 admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:33:20.346192 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6543529-adc7-421c-adc6-b9ee26de80f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:33:20.346852 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6543529-adc7-421c-adc6-b9ee26de80f1 admin admin] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:33:20.347137 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6543529-adc7-421c-adc6-b9ee26de80f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:33:20.347410 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6543529-adc7-421c-adc6-b9ee26de80f1 admin admin] PATH_INFO: `/v3/users` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:33:20.348105 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6543529-adc7-421c-adc6-b9ee26de80f1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:33:20.349149 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: WARNING py.warnings [None req-b6543529-adc7-421c-adc6-b9ee26de80f1 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 25 05:33:20.349149 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: warnings.warn(msg) Jul 25 05:33:20.349149 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: Jul 25 05:33:20.350346 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6543529-adc7-421c-adc6-b9ee26de80f1 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:33:20.389772 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 742/1498] 10.222.0.33 () {60 vars in 1143 bytes} [Mon Jul 25 05:33:20 2022] POST /identity/v3/users => generated 290 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 05:33:20.393826 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2126496c-3c10-4d18-8a23-decacf6d2fbf admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:33:20.395351 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-2126496c-3c10-4d18-8a23-decacf6d2fbf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:33:20.456832 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2126496c-3c10-4d18-8a23-decacf6d2fbf admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:33:20.539076 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:33:20.551321 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d0288c-63b9-4869-af4e-dc208d0555cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:33:20.552064 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d0288c-63b9-4869-af4e-dc208d0555cf admin admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:33:20.552349 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d0288c-63b9-4869-af4e-dc208d0555cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:33:20.552625 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d0288c-63b9-4869-af4e-dc208d0555cf admin admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:33:20.553393 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92d0288c-63b9-4869-af4e-dc208d0555cf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:33:20.554412 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: WARNING py.warnings [None req-92d0288c-63b9-4869-af4e-dc208d0555cf admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 25 05:33:20.554412 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: warnings.warn(msg) Jul 25 05:33:20.554412 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: Jul 25 05:33:20.555292 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92d0288c-63b9-4869-af4e-dc208d0555cf admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:33:20.569381 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 757/1499] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jul 25 05:33:20 2022] GET /identity/v3/roles => generated 1420 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 25 05:33:20.573637 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6543529-adc7-421c-adc6-b9ee26de80f1 admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:33:20.575231 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-b6543529-adc7-421c-adc6-b9ee26de80f1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:33:20.586742 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6543529-adc7-421c-adc6-b9ee26de80f1 admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:33:20.612535 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:33:20.682005 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f37360d-4d60-4c64-8f7c-27513d3f1746 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:33:20.682685 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f37360d-4d60-4c64-8f7c-27513d3f1746 admin admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:33:20.682958 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f37360d-4d60-4c64-8f7c-27513d3f1746 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:33:20.683234 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f37360d-4d60-4c64-8f7c-27513d3f1746 admin admin] PATH_INFO: `/v3/roles` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:33:20.683960 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f37360d-4d60-4c64-8f7c-27513d3f1746 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:33:20.684994 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: WARNING py.warnings [None req-9f37360d-4d60-4c64-8f7c-27513d3f1746 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 25 05:33:20.684994 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: warnings.warn(msg) Jul 25 05:33:20.684994 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: Jul 25 05:33:20.685875 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f37360d-4d60-4c64-8f7c-27513d3f1746 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:33:20.698135 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 743/1500] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jul 25 05:33:20 2022] GET /identity/v3/roles => generated 1420 bytes in 125 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 05:33:20.702420 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d0288c-63b9-4869-af4e-dc208d0555cf admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:33:20.704280 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-92d0288c-63b9-4869-af4e-dc208d0555cf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:33:20.739581 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d0288c-63b9-4869-af4e-dc208d0555cf admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:33:20.832401 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87970bf4-1bb5-433d-959d-620643a63b4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:33:20.833322 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87970bf4-1bb5-433d-959d-620643a63b4b admin admin] REQUEST_METHOD: `PUT` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:33:20.833609 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87970bf4-1bb5-433d-959d-620643a63b4b admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:33:20.833889 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87970bf4-1bb5-433d-959d-620643a63b4b admin admin] PATH_INFO: `/v3/projects/0c3cec01c0294670880345d69b4e890e/users/42e7933e8b75482fa0b0eb3cb7299a48/roles/5074e74e35f54d4b9bb415c34999a82a` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:33:20.857180 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87970bf4-1bb5-433d-959d-620643a63b4b admin admin] RBAC: Authorizing `identity:create_grant(project_id=0c3cec01c0294670880345d69b4e890e, user_id=42e7933e8b75482fa0b0eb3cb7299a48, role_id=5074e74e35f54d4b9bb415c34999a82a)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:33:20.858235 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: WARNING py.warnings [None req-87970bf4-1bb5-433d-959d-620643a63b4b admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 25 05:33:20.858235 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: warnings.warn(msg) Jul 25 05:33:20.858235 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: Jul 25 05:33:20.860211 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87970bf4-1bb5-433d-959d-620643a63b4b admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:33:20.885764 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 758/1501] 10.222.0.33 () {60 vars in 1482 bytes} [Mon Jul 25 05:33:20 2022] PUT /identity/v3/projects/0c3cec01c0294670880345d69b4e890e/users/42e7933e8b75482fa0b0eb3cb7299a48/roles/5074e74e35f54d4b9bb415c34999a82a => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 05:33:20.897933 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5d8686-edc8-422e-b615-774a8e8eae9c None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:33:20.898225 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5d8686-edc8-422e-b615-774a8e8eae9c None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:33:20.898507 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5d8686-edc8-422e-b615-774a8e8eae9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:33:20.955117 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-4d5d8686-edc8-422e-b615-774a8e8eae9c None None] MFA Rules not processed for user `42e7933e8b75482fa0b0eb3cb7299a48`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:33:20.956318 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-4d5d8686-edc8-422e-b615-774a8e8eae9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:33:21.032087 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 744/1502] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 25 05:33:20 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:33:21.045360 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509f5985-f804-4a94-bfa0-22146972b461 None None] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:33:21.045653 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509f5985-f804-4a94-bfa0-22146972b461 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:33:21.045933 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509f5985-f804-4a94-bfa0-22146972b461 None None] PATH_INFO: `/` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:33:21.047314 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 759/1503] 10.222.0.33 () {54 vars in 964 bytes} [Mon Jul 25 05:33:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:33:21.052173 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b91b58-d99b-428e-a895-251d9a7bbf71 None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:33:21.052462 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b91b58-d99b-428e-a895-251d9a7bbf71 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:33:21.052739 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b91b58-d99b-428e-a895-251d9a7bbf71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:33:21.084052 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:33:21.127628 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-d9b91b58-d99b-428e-a895-251d9a7bbf71 None None] MFA Rules not processed for user `db57b478c25643dbaf936f90f6e17d0a`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:33:21.128854 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-d9b91b58-d99b-428e-a895-251d9a7bbf71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:33:21.204011 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 745/1504] 10.222.0.33 () {58 vars in 1061 bytes} [Mon Jul 25 05:33:21 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:33:21.211572 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-509f5985-f804-4a94-bfa0-22146972b461 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:33:21.213132 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-509f5985-f804-4a94-bfa0-22146972b461 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:33:21.263235 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:33:21.316891 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-509f5985-f804-4a94-bfa0-22146972b461 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:33:21.389552 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94deea71-2828-4a38-9aad-f95c342a1b20 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db57b478c25643dbaf936f90f6e17d0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92afba266cce4d1bad4134c351e74be3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:33:21.390133 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94deea71-2828-4a38-9aad-f95c342a1b20 service cinder] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:33:21.390419 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94deea71-2828-4a38-9aad-f95c342a1b20 service cinder] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:33:21.390694 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94deea71-2828-4a38-9aad-f95c342a1b20 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:33:21.392331 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-94deea71-2828-4a38-9aad-f95c342a1b20 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:33:21.489873 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94deea71-2828-4a38-9aad-f95c342a1b20 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:33:21.491290 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94deea71-2828-4a38-9aad-f95c342a1b20 service cinder] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:33:21.495295 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:33:21.510032 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 760/1505] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jul 25 05:33:21 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 299 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 05:33:21.551873 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b189ab-f3ab-4743-a516-204f13fa1477 None None] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:33:21.552164 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b189ab-f3ab-4743-a516-204f13fa1477 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:33:21.552439 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b189ab-f3ab-4743-a516-204f13fa1477 None None] PATH_INFO: `/` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:33:21.553761 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 746/1506] 10.222.0.33 () {52 vars in 829 bytes} [Mon Jul 25 05:33:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:33:21.557739 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94deea71-2828-4a38-9aad-f95c342a1b20 service cinder] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:33:21.558996 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-94deea71-2828-4a38-9aad-f95c342a1b20 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:33:21.658109 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94deea71-2828-4a38-9aad-f95c342a1b20 service cinder] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:33:21.750579 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf8666e-32a7-4ed3-a867-3697a3273d94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:33:21.751437 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf8666e-32a7-4ed3-a867-3697a3273d94 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:33:21.751716 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf8666e-32a7-4ed3-a867-3697a3273d94 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:33:21.751988 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf8666e-32a7-4ed3-a867-3697a3273d94 admin admin] PATH_INFO: `/v3/users/42e7933e8b75482fa0b0eb3cb7299a48` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:33:21.764577 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf8666e-32a7-4ed3-a867-3697a3273d94 admin admin] RBAC: Authorizing `identity:delete_user(user_id=42e7933e8b75482fa0b0eb3cb7299a48)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:33:21.765630 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: WARNING py.warnings [None req-2cf8666e-32a7-4ed3-a867-3697a3273d94 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 25 05:33:21.765630 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: warnings.warn(msg) Jul 25 05:33:21.765630 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: Jul 25 05:33:21.767205 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf8666e-32a7-4ed3-a867-3697a3273d94 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:33:21.777308 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:33:21.831328 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-2cf8666e-32a7-4ed3-a867-3697a3273d94 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '42e7933e8b75482fa0b0eb3cb7299a48'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:33:21.831861 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-2cf8666e-32a7-4ed3-a867-3697a3273d94 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '42e7933e8b75482fa0b0eb3cb7299a48'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:33:21.844575 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-2cf8666e-32a7-4ed3-a867-3697a3273d94 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '42e7933e8b75482fa0b0eb3cb7299a48'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:33:21.862174 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-2cf8666e-32a7-4ed3-a867-3697a3273d94 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '42e7933e8b75482fa0b0eb3cb7299a48'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:33:21.862621 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-2cf8666e-32a7-4ed3-a867-3697a3273d94 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '42e7933e8b75482fa0b0eb3cb7299a48'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:33:21.876861 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 761/1507] 10.222.0.33 () {58 vars in 1223 bytes} [Mon Jul 25 05:33:21 2022] DELETE /identity/v3/users/42e7933e8b75482fa0b0eb3cb7299a48 => generated 0 bytes in 320 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 05:33:21.881320 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a479436-daaa-4eb1-972c-1232616ddcc0 None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:33:21.881779 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a479436-daaa-4eb1-972c-1232616ddcc0 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:33:21.881895 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a479436-daaa-4eb1-972c-1232616ddcc0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:33:21.936730 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-5a479436-daaa-4eb1-972c-1232616ddcc0 None None] MFA Rules not processed for user `1eae540242d94c10b0ad33960a855f08`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:33:21.937900 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-5a479436-daaa-4eb1-972c-1232616ddcc0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:33:22.012193 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:33:22.025351 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 747/1508] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 25 05:33:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:33:22.035519 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf8666e-32a7-4ed3-a867-3697a3273d94 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:33:22.036721 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-2cf8666e-32a7-4ed3-a867-3697a3273d94 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:33:22.134433 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf8666e-32a7-4ed3-a867-3697a3273d94 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:33:22.189896 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:33:22.210726 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b79289-18f7-4105-a8e0-0106b918f77e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b2931200f99e43d79a50a9f6e02738e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92afba266cce4d1bad4134c351e74be3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:33:22.211387 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b79289-18f7-4105-a8e0-0106b918f77e service neutron] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:33:22.211675 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b79289-18f7-4105-a8e0-0106b918f77e service neutron] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:33:22.211955 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b79289-18f7-4105-a8e0-0106b918f77e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:33:22.213649 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-c2b79289-18f7-4105-a8e0-0106b918f77e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:33:22.320142 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2b79289-18f7-4105-a8e0-0106b918f77e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:33:22.321599 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2b79289-18f7-4105-a8e0-0106b918f77e service neutron] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:33:22.340532 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 762/1509] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jul 25 05:33:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 05:33:22.696239 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a479436-daaa-4eb1-972c-1232616ddcc0 None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:33:22.697498 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-5a479436-daaa-4eb1-972c-1232616ddcc0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:33:22.709150 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a479436-daaa-4eb1-972c-1232616ddcc0 None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:33:22.809090 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca841346-f4c3-4109-b384-615dcade4bf6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:33:22.809961 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca841346-f4c3-4109-b384-615dcade4bf6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:33:22.810246 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca841346-f4c3-4109-b384-615dcade4bf6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:33:22.810523 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca841346-f4c3-4109-b384-615dcade4bf6 admin admin] PATH_INFO: `/v3/projects/0c3cec01c0294670880345d69b4e890e` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:33:22.820507 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca841346-f4c3-4109-b384-615dcade4bf6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0c3cec01c0294670880345d69b4e890e)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:33:22.821588 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: WARNING py.warnings [None req-ca841346-f4c3-4109-b384-615dcade4bf6 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 25 05:33:22.821588 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: warnings.warn(msg) Jul 25 05:33:22.821588 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: Jul 25 05:33:22.822854 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca841346-f4c3-4109-b384-615dcade4bf6 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:33:22.881289 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-ca841346-f4c3-4109-b384-615dcade4bf6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0c3cec01c0294670880345d69b4e890e', 'request_id': 'req-ca841346-f4c3-4109-b384-615dcade4bf6'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:33:22.887352 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-ca841346-f4c3-4109-b384-615dcade4bf6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0c3cec01c0294670880345d69b4e890e', 'request_id': 'req-ca841346-f4c3-4109-b384-615dcade4bf6'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:33:22.900242 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-ca841346-f4c3-4109-b384-615dcade4bf6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0c3cec01c0294670880345d69b4e890e', 'request_id': 'req-ca841346-f4c3-4109-b384-615dcade4bf6'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:33:22.926404 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-ca841346-f4c3-4109-b384-615dcade4bf6 admin admin] The token cache is being invalidate because project 0c3cec01c0294670880345d69b4e890e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80248) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 05:33:22.926701 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-ca841346-f4c3-4109-b384-615dcade4bf6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:33:22.927200 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-ca841346-f4c3-4109-b384-615dcade4bf6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:33:22.928544 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 748/1510] 10.222.0.33 () {58 vars in 1232 bytes} [Mon Jul 25 05:33:22 2022] DELETE /identity/v3/projects/0c3cec01c0294670880345d69b4e890e => generated 0 bytes in 233 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 05:33:32.338284 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=62 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:33:32.924478 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=84,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:35:04.266229 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e860fb3-2ad5-4040-b96d-e9036a2e8ab2 None None] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:35:04.266810 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e860fb3-2ad5-4040-b96d-e9036a2e8ab2 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:35:04.266810 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e860fb3-2ad5-4040-b96d-e9036a2e8ab2 None None] PATH_INFO: `/` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:35:04.268101 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 763/1511] 127.0.0.1 () {54 vars in 901 bytes} [Mon Jul 25 05:35:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:35:04.273156 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8732973b-f90b-407d-bc31-9a67dfb698f9 None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:35:04.274129 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8732973b-f90b-407d-bc31-9a67dfb698f9 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:35:04.274431 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8732973b-f90b-407d-bc31-9a67dfb698f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:35:04.352354 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-8732973b-f90b-407d-bc31-9a67dfb698f9 None None] MFA Rules not processed for user `1eae540242d94c10b0ad33960a855f08`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:35:04.353573 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-8732973b-f90b-407d-bc31-9a67dfb698f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:35:04.455834 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 749/1512] 127.0.0.1 () {58 vars in 998 bytes} [Mon Jul 25 05:35:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:35:04.607679 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e860fb3-2ad5-4040-b96d-e9036a2e8ab2 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:35:04.610089 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-7e860fb3-2ad5-4040-b96d-e9036a2e8ab2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:35:04.728106 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e860fb3-2ad5-4040-b96d-e9036a2e8ab2 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:35:04.733652 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c0149f-7835-4588-bbf9-bcbdc9d8d57d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cd482a61e8a420db45bb532e5dae6e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92afba266cce4d1bad4134c351e74be3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:35:04.734307 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c0149f-7835-4588-bbf9-bcbdc9d8d57d service nova] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:35:04.734598 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c0149f-7835-4588-bbf9-bcbdc9d8d57d service nova] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:35:04.734881 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c0149f-7835-4588-bbf9-bcbdc9d8d57d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:35:04.743043 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1c0149f-7835-4588-bbf9-bcbdc9d8d57d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:35:04.744628 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1c0149f-7835-4588-bbf9-bcbdc9d8d57d service nova] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:35:04.754017 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 764/1513] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jul 25 05:35:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 05:35:05.601553 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e1828d-c55c-421e-8a14-97bc96128cc3 None None] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:35:05.602870 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e1828d-c55c-421e-8a14-97bc96128cc3 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:35:05.602988 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e1828d-c55c-421e-8a14-97bc96128cc3 None None] PATH_INFO: `/` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:35:05.604400 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 750/1514] 127.0.0.1 () {54 vars in 901 bytes} [Mon Jul 25 05:35:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:35:05.609225 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3817a7ab-0207-44ec-ab4f-0c1684a64e52 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:35:05.609771 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3817a7ab-0207-44ec-ab4f-0c1684a64e52 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:35:05.609771 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3817a7ab-0207-44ec-ab4f-0c1684a64e52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:35:05.627564 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-3817a7ab-0207-44ec-ab4f-0c1684a64e52 None None] MFA Rules not processed for user `1eae540242d94c10b0ad33960a855f08`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:35:05.628753 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-3817a7ab-0207-44ec-ab4f-0c1684a64e52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:35:05.636650 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 765/1515] 127.0.0.1 () {58 vars in 998 bytes} [Mon Jul 25 05:35:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:35:05.747242 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82c6c56-9520-4e49-9d49-095b8023ad0e None None] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:35:05.747537 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82c6c56-9520-4e49-9d49-095b8023ad0e None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:35:05.747824 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82c6c56-9520-4e49-9d49-095b8023ad0e None None] PATH_INFO: `/v3` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:35:05.749267 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 751/1516] 10.222.0.43 () {54 vars in 925 bytes} [Mon Jul 25 05:35:05 2022] GET /identity/v3 => generated 255 bytes in 3 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:35:05.755997 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c5dc1f-375a-4b88-b393-83e47529cbba None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:35:05.756294 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c5dc1f-375a-4b88-b393-83e47529cbba None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:35:05.756578 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c5dc1f-375a-4b88-b393-83e47529cbba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:35:05.787527 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b82c6c56-9520-4e49-9d49-095b8023ad0e None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:35:05.803347 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b82c6c56-9520-4e49-9d49-095b8023ad0e None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:35:05.809130 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a51cd86-cdd5-488b-a99c-5d0d9fdd3b80 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cd482a61e8a420db45bb532e5dae6e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92afba266cce4d1bad4134c351e74be3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:35:05.809902 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a51cd86-cdd5-488b-a99c-5d0d9fdd3b80 service nova] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:35:05.810196 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a51cd86-cdd5-488b-a99c-5d0d9fdd3b80 service nova] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:35:05.810474 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a51cd86-cdd5-488b-a99c-5d0d9fdd3b80 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:35:05.819071 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a51cd86-cdd5-488b-a99c-5d0d9fdd3b80 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:35:05.821236 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-76c5dc1f-375a-4b88-b393-83e47529cbba None None] MFA Rules not processed for user `317790016e7047fe96af73925de5d872`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:35:05.821453 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a51cd86-cdd5-488b-a99c-5d0d9fdd3b80 service nova] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:35:05.822461 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-76c5dc1f-375a-4b88-b393-83e47529cbba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:35:05.831774 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 752/1517] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jul 25 05:35:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 25 05:35:05.877265 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:35:05.898368 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 766/1518] 10.222.0.43 () {58 vars in 1015 bytes} [Mon Jul 25 05:35:05 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:35:14.884300 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6887a7cf-f611-49d7-b2f1-833a8f65a91e None None] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:35:14.884851 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6887a7cf-f611-49d7-b2f1-833a8f65a91e None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:35:14.884964 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6887a7cf-f611-49d7-b2f1-833a8f65a91e None None] PATH_INFO: `/` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:35:14.886790 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 753/1519] 10.222.0.33 () {54 vars in 961 bytes} [Mon Jul 25 05:35:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:35:14.891795 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbbebb44-0f69-4edb-b56a-b18b41029535 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:35:14.892199 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbbebb44-0f69-4edb-b56a-b18b41029535 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:35:14.892364 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbbebb44-0f69-4edb-b56a-b18b41029535 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:35:14.910791 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-cbbebb44-0f69-4edb-b56a-b18b41029535 None None] MFA Rules not processed for user `317790016e7047fe96af73925de5d872`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:35:14.912003 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-cbbebb44-0f69-4edb-b56a-b18b41029535 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:35:14.920684 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 767/1520] 10.222.0.33 () {58 vars in 1058 bytes} [Mon Jul 25 05:35:14 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:35:14.928037 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6887a7cf-f611-49d7-b2f1-833a8f65a91e None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:35:14.941169 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6887a7cf-f611-49d7-b2f1-833a8f65a91e None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:35:14.946130 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5aab8a7-9543-4abe-becb-3650fd833603 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '317790016e7047fe96af73925de5d872', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92afba266cce4d1bad4134c351e74be3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:35:14.946764 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5aab8a7-9543-4abe-becb-3650fd833603 service placement] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:35:14.947163 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5aab8a7-9543-4abe-becb-3650fd833603 service placement] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:35:14.947451 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5aab8a7-9543-4abe-becb-3650fd833603 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:35:14.956378 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5aab8a7-9543-4abe-becb-3650fd833603 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:35:14.957997 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5aab8a7-9543-4abe-becb-3650fd833603 service placement] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:35:14.968145 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 754/1521] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jul 25 05:35:14 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 05:35:17.075148 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474004aa-3c13-4f10-9caa-778ba8312915 None None] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:35:17.075688 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474004aa-3c13-4f10-9caa-778ba8312915 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:35:17.075688 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474004aa-3c13-4f10-9caa-778ba8312915 None None] PATH_INFO: `/` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:35:17.077314 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 768/1522] 127.0.0.1 () {54 vars in 901 bytes} [Mon Jul 25 05:35:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:35:17.082048 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca6c794-7040-4ac8-bc05-0ff18926ec6e None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:35:17.082458 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca6c794-7040-4ac8-bc05-0ff18926ec6e None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:35:17.082638 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca6c794-7040-4ac8-bc05-0ff18926ec6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:35:17.100741 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-eca6c794-7040-4ac8-bc05-0ff18926ec6e None None] MFA Rules not processed for user `1eae540242d94c10b0ad33960a855f08`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:35:17.101981 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-eca6c794-7040-4ac8-bc05-0ff18926ec6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:35:17.110707 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 755/1523] 127.0.0.1 () {58 vars in 998 bytes} [Mon Jul 25 05:35:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:35:17.263986 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-474004aa-3c13-4f10-9caa-778ba8312915 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:35:17.278807 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-474004aa-3c13-4f10-9caa-778ba8312915 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:35:17.284100 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe747b1-ebb3-475c-905f-e8eb9227e2a8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cd482a61e8a420db45bb532e5dae6e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92afba266cce4d1bad4134c351e74be3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:35:17.284733 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe747b1-ebb3-475c-905f-e8eb9227e2a8 service nova] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:35:17.285044 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe747b1-ebb3-475c-905f-e8eb9227e2a8 service nova] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:35:17.285320 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe747b1-ebb3-475c-905f-e8eb9227e2a8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:35:17.293865 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fe747b1-ebb3-475c-905f-e8eb9227e2a8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:35:17.295433 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fe747b1-ebb3-475c-905f-e8eb9227e2a8 service nova] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:35:17.304989 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 769/1524] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jul 25 05:35:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 05:35:18.142937 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ad6ee9-f84a-4c38-8496-a30ee4e4ae12 None None] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:35:18.143491 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ad6ee9-f84a-4c38-8496-a30ee4e4ae12 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:35:18.143491 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ad6ee9-f84a-4c38-8496-a30ee4e4ae12 None None] PATH_INFO: `/` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:35:18.144929 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 756/1525] 127.0.0.1 () {54 vars in 901 bytes} [Mon Jul 25 05:35:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:35:18.149696 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7f4f8e-5869-4c4e-9889-dd8995f73fad None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:35:18.150144 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7f4f8e-5869-4c4e-9889-dd8995f73fad None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:35:18.150323 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7f4f8e-5869-4c4e-9889-dd8995f73fad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:35:18.169939 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-0c7f4f8e-5869-4c4e-9889-dd8995f73fad None None] MFA Rules not processed for user `1eae540242d94c10b0ad33960a855f08`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:35:18.171126 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-0c7f4f8e-5869-4c4e-9889-dd8995f73fad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:35:18.178864 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 770/1526] 127.0.0.1 () {58 vars in 998 bytes} [Mon Jul 25 05:35:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:35:18.328591 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ad6ee9-f84a-4c38-8496-a30ee4e4ae12 None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:35:18.343592 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ad6ee9-f84a-4c38-8496-a30ee4e4ae12 None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:35:18.348977 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92616ae7-ee3d-4282-b8fb-ed4a34d6dda8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cd482a61e8a420db45bb532e5dae6e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92afba266cce4d1bad4134c351e74be3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:35:18.349609 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92616ae7-ee3d-4282-b8fb-ed4a34d6dda8 service nova] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:35:18.349893 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92616ae7-ee3d-4282-b8fb-ed4a34d6dda8 service nova] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:35:18.350167 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92616ae7-ee3d-4282-b8fb-ed4a34d6dda8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:35:18.358918 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92616ae7-ee3d-4282-b8fb-ed4a34d6dda8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:35:18.360486 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92616ae7-ee3d-4282-b8fb-ed4a34d6dda8 service nova] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:35:18.369717 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 757/1527] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jul 25 05:35:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 05:35:22.407775 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93619d1-af7f-4f13-9607-eea4cb01cd52 None None] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:35:22.408368 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93619d1-af7f-4f13-9607-eea4cb01cd52 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:35:22.408368 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93619d1-af7f-4f13-9607-eea4cb01cd52 None None] PATH_INFO: `/` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:35:22.409762 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 771/1528] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jul 25 05:35:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:35:22.414924 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2bb106e-ead8-409f-98dd-b6a74ff94dff None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:35:22.415342 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2bb106e-ead8-409f-98dd-b6a74ff94dff None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:35:22.415493 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2bb106e-ead8-409f-98dd-b6a74ff94dff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:35:22.434935 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-f2bb106e-ead8-409f-98dd-b6a74ff94dff None None] MFA Rules not processed for user `1eae540242d94c10b0ad33960a855f08`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:35:22.436178 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-f2bb106e-ead8-409f-98dd-b6a74ff94dff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:35:22.444219 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 758/1529] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jul 25 05:35:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:35:22.517763 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f93619d1-af7f-4f13-9607-eea4cb01cd52 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:35:22.519480 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-f93619d1-af7f-4f13-9607-eea4cb01cd52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:35:22.598071 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f93619d1-af7f-4f13-9607-eea4cb01cd52 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:35:22.601635 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc1ac6a-769c-447a-bed4-1045164f2aa8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b2931200f99e43d79a50a9f6e02738e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92afba266cce4d1bad4134c351e74be3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:35:22.602276 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc1ac6a-769c-447a-bed4-1045164f2aa8 service neutron] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:35:22.602559 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc1ac6a-769c-447a-bed4-1045164f2aa8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:35:22.602836 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc1ac6a-769c-447a-bed4-1045164f2aa8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:35:22.612152 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cc1ac6a-769c-447a-bed4-1045164f2aa8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:35:22.613594 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cc1ac6a-769c-447a-bed4-1045164f2aa8 service neutron] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:35:22.622918 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 772/1530] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jul 25 05:35:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 05:35:24.655493 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b52b80b-c258-4fd7-b394-af16ad8894c9 None None] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:35:24.655974 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b52b80b-c258-4fd7-b394-af16ad8894c9 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:35:24.656072 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b52b80b-c258-4fd7-b394-af16ad8894c9 None None] PATH_INFO: `/` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:35:24.657479 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 759/1531] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jul 25 05:35:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:35:24.662648 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d4aa4e-2329-47d4-af6c-9b0f9d42e777 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:35:24.663076 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d4aa4e-2329-47d4-af6c-9b0f9d42e777 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:35:24.663234 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d4aa4e-2329-47d4-af6c-9b0f9d42e777 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:35:24.681199 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-d1d4aa4e-2329-47d4-af6c-9b0f9d42e777 None None] MFA Rules not processed for user `1eae540242d94c10b0ad33960a855f08`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:35:24.682386 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-d1d4aa4e-2329-47d4-af6c-9b0f9d42e777 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:35:24.690348 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 773/1532] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jul 25 05:35:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:35:24.727297 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b52b80b-c258-4fd7-b394-af16ad8894c9 None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:35:24.738041 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b52b80b-c258-4fd7-b394-af16ad8894c9 None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:35:24.741606 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb99a1f5-e9c1-44c5-9881-23f6e001fe57 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b2931200f99e43d79a50a9f6e02738e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92afba266cce4d1bad4134c351e74be3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:35:24.742241 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb99a1f5-e9c1-44c5-9881-23f6e001fe57 service neutron] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:35:24.742527 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb99a1f5-e9c1-44c5-9881-23f6e001fe57 service neutron] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:35:24.742806 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb99a1f5-e9c1-44c5-9881-23f6e001fe57 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:35:24.751966 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb99a1f5-e9c1-44c5-9881-23f6e001fe57 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:35:24.753434 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb99a1f5-e9c1-44c5-9881-23f6e001fe57 service neutron] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:35:24.763496 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 760/1533] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jul 25 05:35:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 05:35:28.441073 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0f0b87-8624-4cc4-b07f-e51684f56168 None None] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:35:28.441702 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0f0b87-8624-4cc4-b07f-e51684f56168 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:35:28.441702 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0f0b87-8624-4cc4-b07f-e51684f56168 None None] PATH_INFO: `/` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:35:28.443028 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 774/1534] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jul 25 05:35:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:35:28.449129 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56f9a62-6dde-4b1b-b554-a9fab5eb13fa None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:35:28.449659 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56f9a62-6dde-4b1b-b554-a9fab5eb13fa None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:35:28.449880 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56f9a62-6dde-4b1b-b554-a9fab5eb13fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:35:28.456211 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:35:28.471490 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-f56f9a62-6dde-4b1b-b554-a9fab5eb13fa None None] MFA Rules not processed for user `1eae540242d94c10b0ad33960a855f08`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:35:28.472695 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-f56f9a62-6dde-4b1b-b554-a9fab5eb13fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:35:28.481661 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 761/1535] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jul 25 05:35:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 05:35:28.783763 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb0f0b87-8624-4cc4-b07f-e51684f56168 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:35:28.785478 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-fb0f0b87-8624-4cc4-b07f-e51684f56168 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:35:28.866386 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb0f0b87-8624-4cc4-b07f-e51684f56168 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:35:28.869915 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85383da4-799a-43c9-8bc4-0d9e96ec88d7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db57b478c25643dbaf936f90f6e17d0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92afba266cce4d1bad4134c351e74be3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:35:28.871149 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85383da4-799a-43c9-8bc4-0d9e96ec88d7 service cinder] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:35:28.871430 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85383da4-799a-43c9-8bc4-0d9e96ec88d7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:35:28.871703 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85383da4-799a-43c9-8bc4-0d9e96ec88d7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:35:28.880509 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85383da4-799a-43c9-8bc4-0d9e96ec88d7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:35:28.881966 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85383da4-799a-43c9-8bc4-0d9e96ec88d7 service cinder] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:35:28.891763 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 775/1536] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jul 25 05:35:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 05:35:30.958281 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57261c3b-13a9-4bf2-bea6-d015550762dd None None] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:35:30.958865 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57261c3b-13a9-4bf2-bea6-d015550762dd None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:35:30.958865 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57261c3b-13a9-4bf2-bea6-d015550762dd None None] PATH_INFO: `/` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:35:30.960262 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 762/1537] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jul 25 05:35:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:35:30.965814 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b7d329-3b2b-4a4d-b219-4620f58d4a44 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:35:30.966313 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b7d329-3b2b-4a4d-b219-4620f58d4a44 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:35:30.966417 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b7d329-3b2b-4a4d-b219-4620f58d4a44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:35:30.984825 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-70b7d329-3b2b-4a4d-b219-4620f58d4a44 None None] MFA Rules not processed for user `1eae540242d94c10b0ad33960a855f08`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:35:30.986028 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-70b7d329-3b2b-4a4d-b219-4620f58d4a44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:35:30.993941 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 776/1538] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jul 25 05:35:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 05:35:31.301507 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57261c3b-13a9-4bf2-bea6-d015550762dd None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:35:31.303187 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-57261c3b-13a9-4bf2-bea6-d015550762dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:35:31.316025 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57261c3b-13a9-4bf2-bea6-d015550762dd None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:35:31.319555 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-385e00da-b4ba-4f45-96a0-36b198e17b96 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db57b478c25643dbaf936f90f6e17d0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92afba266cce4d1bad4134c351e74be3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:35:31.320191 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385e00da-b4ba-4f45-96a0-36b198e17b96 service cinder] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:35:31.320474 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385e00da-b4ba-4f45-96a0-36b198e17b96 service cinder] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:35:31.320754 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385e00da-b4ba-4f45-96a0-36b198e17b96 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:35:31.330426 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-385e00da-b4ba-4f45-96a0-36b198e17b96 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:35:31.331831 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-385e00da-b4ba-4f45-96a0-36b198e17b96 service cinder] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:35:31.342257 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 763/1539] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jul 25 05:35:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 05:35:40.979005 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=92 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:35:41.337018 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=8 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:37:00.773843 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e96712d-38b0-4468-964d-2745b5c3fc55 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:37:00.774500 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e96712d-38b0-4468-964d-2745b5c3fc55 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:37:00.774500 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e96712d-38b0-4468-964d-2745b5c3fc55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:37:00.810229 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-3e96712d-38b0-4468-964d-2745b5c3fc55 None None] MFA Rules not processed for user `1eae540242d94c10b0ad33960a855f08`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:37:00.811481 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-3e96712d-38b0-4468-964d-2745b5c3fc55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:37:00.837897 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 777/1540] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 25 05:37:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 65 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 05:37:00.847845 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f5a6f2-986d-49ec-9571-0c093615499e None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:37:00.848451 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f5a6f2-986d-49ec-9571-0c093615499e None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:37:00.848451 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f5a6f2-986d-49ec-9571-0c093615499e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:37:00.877888 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-73f5a6f2-986d-49ec-9571-0c093615499e None None] MFA Rules not processed for user `1eae540242d94c10b0ad33960a855f08`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:37:00.879121 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-73f5a6f2-986d-49ec-9571-0c093615499e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:37:00.899800 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 764/1541] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 25 05:37:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:37:00.904172 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e96712d-38b0-4468-964d-2745b5c3fc55 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:37:00.914347 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e96712d-38b0-4468-964d-2745b5c3fc55 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:37:00.925329 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a73ff3c-5b9d-4a15-8fc1-243948244816 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:37:00.926015 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a73ff3c-5b9d-4a15-8fc1-243948244816 admin admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:37:00.926310 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a73ff3c-5b9d-4a15-8fc1-243948244816 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:37:00.926594 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a73ff3c-5b9d-4a15-8fc1-243948244816 admin admin] PATH_INFO: `/v3/domains` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:37:00.927343 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a73ff3c-5b9d-4a15-8fc1-243948244816 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:37:00.927658 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a73ff3c-5b9d-4a15-8fc1-243948244816 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:37:00.928690 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: WARNING py.warnings [None req-6a73ff3c-5b9d-4a15-8fc1-243948244816 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 25 05:37:00.928690 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: warnings.warn(msg) Jul 25 05:37:00.928690 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: Jul 25 05:37:00.937787 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a73ff3c-5b9d-4a15-8fc1-243948244816 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:37:00.951777 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 778/1542] 10.222.0.33 () {58 vars in 1152 bytes} [Mon Jul 25 05:37:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:37:00.960031 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f5a6f2-986d-49ec-9571-0c093615499e None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:37:00.973888 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f5a6f2-986d-49ec-9571-0c093615499e None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:37:00.986208 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb8a93b-7c0c-4286-8218-8e48bda45cd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:37:00.986886 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb8a93b-7c0c-4286-8218-8e48bda45cd2 admin admin] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:37:00.987170 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb8a93b-7c0c-4286-8218-8e48bda45cd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:37:00.987444 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb8a93b-7c0c-4286-8218-8e48bda45cd2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:37:00.988138 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeb8a93b-7c0c-4286-8218-8e48bda45cd2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:37:00.989324 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: WARNING py.warnings [None req-aeb8a93b-7c0c-4286-8218-8e48bda45cd2 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 25 05:37:00.989324 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: warnings.warn(msg) Jul 25 05:37:00.989324 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: Jul 25 05:37:00.990625 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeb8a93b-7c0c-4286-8218-8e48bda45cd2 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:37:01.061439 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f9b868-7972-47f9-a41b-95bbbc582703 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:37:01.062108 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f9b868-7972-47f9-a41b-95bbbc582703 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:37:01.062255 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 765/1543] 10.222.0.33 () {60 vars in 1152 bytes} [Mon Jul 25 05:37:00 2022] POST /identity/v3/projects => generated 393 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 05:37:01.062772 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f9b868-7972-47f9-a41b-95bbbc582703 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:37:01.068365 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb8a93b-7c0c-4286-8218-8e48bda45cd2 admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:37:01.096246 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-84f9b868-7972-47f9-a41b-95bbbc582703 None None] MFA Rules not processed for user `1eae540242d94c10b0ad33960a855f08`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:37:01.097990 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-84f9b868-7972-47f9-a41b-95bbbc582703 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:37:01.173498 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb8a93b-7c0c-4286-8218-8e48bda45cd2 admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:37:01.180023 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e872bdef-8865-44b6-901e-228b970634f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:37:01.180023 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e872bdef-8865-44b6-901e-228b970634f3 admin admin] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:37:01.180293 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e872bdef-8865-44b6-901e-228b970634f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:37:01.180518 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e872bdef-8865-44b6-901e-228b970634f3 admin admin] PATH_INFO: `/v3/users` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:37:01.181311 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e872bdef-8865-44b6-901e-228b970634f3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:37:01.184208 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e872bdef-8865-44b6-901e-228b970634f3 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:37:01.201756 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 779/1544] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 25 05:37:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 05:37:01.203973 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1f1884-788a-4130-aabe-aa412945d13d None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:37:01.204270 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1f1884-788a-4130-aabe-aa412945d13d None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:37:01.204555 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1f1884-788a-4130-aabe-aa412945d13d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:37:01.209293 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 766/1545] 10.222.0.33 () {60 vars in 1143 bytes} [Mon Jul 25 05:37:01 2022] POST /identity/v3/users => generated 297 bytes in 141 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 25 05:37:01.210494 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d968057-d850-407b-851c-0ff01f79041a None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:37:01.210789 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d968057-d850-407b-851c-0ff01f79041a None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:37:01.211073 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d968057-d850-407b-851c-0ff01f79041a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:37:01.233941 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-2d968057-d850-407b-851c-0ff01f79041a None None] MFA Rules not processed for user `1eae540242d94c10b0ad33960a855f08`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:37:01.234616 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-de1f1884-788a-4130-aabe-aa412945d13d None None] MFA Rules not processed for user `1eae540242d94c10b0ad33960a855f08`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:37:01.235178 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-2d968057-d850-407b-851c-0ff01f79041a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:37:01.235807 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-de1f1884-788a-4130-aabe-aa412945d13d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:37:01.244083 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 780/1546] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 25 05:37:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:37:01.244228 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 767/1547] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 25 05:37:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:37:01.245245 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988b1c87-60a8-4a61-a2dd-ef95cb38a152 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:37:01.245379 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9c71ff-4857-431a-b812-5395063a5889 None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:37:01.245610 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9c71ff-4857-431a-b812-5395063a5889 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:37:01.245849 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988b1c87-60a8-4a61-a2dd-ef95cb38a152 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:37:01.245954 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9c71ff-4857-431a-b812-5395063a5889 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:37:01.246040 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988b1c87-60a8-4a61-a2dd-ef95cb38a152 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:37:01.272081 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-988b1c87-60a8-4a61-a2dd-ef95cb38a152 None None] MFA Rules not processed for user `1eae540242d94c10b0ad33960a855f08`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:37:01.273312 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-988b1c87-60a8-4a61-a2dd-ef95cb38a152 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:37:01.274518 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-4a9c71ff-4857-431a-b812-5395063a5889 None None] MFA Rules not processed for user `1eae540242d94c10b0ad33960a855f08`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:37:01.275760 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-4a9c71ff-4857-431a-b812-5395063a5889 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:37:01.282095 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 781/1548] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 25 05:37:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:37:01.283417 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7335ecb2-b4b7-4756-b082-eb0b46fa93c6 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:37:01.283705 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7335ecb2-b4b7-4756-b082-eb0b46fa93c6 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:37:01.283984 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7335ecb2-b4b7-4756-b082-eb0b46fa93c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:37:01.284664 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 768/1549] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 25 05:37:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:37:01.289758 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a9c71ff-4857-431a-b812-5395063a5889 None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:37:01.300378 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a9c71ff-4857-431a-b812-5395063a5889 None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:37:01.305210 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6712fea2-14f5-43de-8a4b-a2c3a2f46a83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:37:01.305530 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-7335ecb2-b4b7-4756-b082-eb0b46fa93c6 None None] MFA Rules not processed for user `1eae540242d94c10b0ad33960a855f08`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:37:01.305823 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6712fea2-14f5-43de-8a4b-a2c3a2f46a83 admin admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:37:01.306114 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6712fea2-14f5-43de-8a4b-a2c3a2f46a83 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:37:01.306391 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6712fea2-14f5-43de-8a4b-a2c3a2f46a83 admin admin] PATH_INFO: `/v3/roles` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:37:01.306729 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-7335ecb2-b4b7-4756-b082-eb0b46fa93c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:37:01.307122 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6712fea2-14f5-43de-8a4b-a2c3a2f46a83 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:37:01.308914 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6712fea2-14f5-43de-8a4b-a2c3a2f46a83 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:37:01.314894 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 782/1550] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 25 05:37:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:37:01.317905 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba33864-c3d4-4428-b44e-4824db77402f None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:37:01.318201 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba33864-c3d4-4428-b44e-4824db77402f None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:37:01.318611 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba33864-c3d4-4428-b44e-4824db77402f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:37:01.320167 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 769/1551] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jul 25 05:37:01 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 05:37:01.321396 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b33630e-ad6c-4854-b84a-f9c45c079da8 None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:37:01.321690 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b33630e-ad6c-4854-b84a-f9c45c079da8 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:37:01.322125 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b33630e-ad6c-4854-b84a-f9c45c079da8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:37:01.340736 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-bba33864-c3d4-4428-b44e-4824db77402f None None] MFA Rules not processed for user `1eae540242d94c10b0ad33960a855f08`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:37:01.341590 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-8b33630e-ad6c-4854-b84a-f9c45c079da8 None None] MFA Rules not processed for user `1eae540242d94c10b0ad33960a855f08`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:37:01.341979 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-bba33864-c3d4-4428-b44e-4824db77402f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:37:01.342840 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-8b33630e-ad6c-4854-b84a-f9c45c079da8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:37:01.349952 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 783/1552] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 25 05:37:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:37:01.353894 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 770/1553] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 25 05:37:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:37:01.354066 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fead324-ccff-4453-9b1f-16977214bf5e None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:37:01.354066 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fead324-ccff-4453-9b1f-16977214bf5e None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:37:01.354066 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fead324-ccff-4453-9b1f-16977214bf5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:37:01.354342 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff89cbf-eadb-4a7a-a09f-6c4e1a9bed7b None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:37:01.354342 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff89cbf-eadb-4a7a-a09f-6c4e1a9bed7b None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:37:01.354342 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff89cbf-eadb-4a7a-a09f-6c4e1a9bed7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:37:01.371841 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-4fead324-ccff-4453-9b1f-16977214bf5e None None] MFA Rules not processed for user `1eae540242d94c10b0ad33960a855f08`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:37:01.372670 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-7ff89cbf-eadb-4a7a-a09f-6c4e1a9bed7b None None] MFA Rules not processed for user `1eae540242d94c10b0ad33960a855f08`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:37:01.373078 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-4fead324-ccff-4453-9b1f-16977214bf5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:37:01.373881 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-7ff89cbf-eadb-4a7a-a09f-6c4e1a9bed7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:37:01.384542 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 784/1554] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 25 05:37:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:37:01.384705 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 771/1555] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 25 05:37:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:37:01.384830 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fbadde0-a1f6-486b-ba86-f074c665e598 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:37:01.384830 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fbadde0-a1f6-486b-ba86-f074c665e598 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:37:01.384830 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fbadde0-a1f6-486b-ba86-f074c665e598 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:37:01.385110 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c124e93-9a00-459b-ad8b-b86faa907ca4 None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:37:01.385110 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c124e93-9a00-459b-ad8b-b86faa907ca4 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:37:01.385110 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c124e93-9a00-459b-ad8b-b86faa907ca4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:37:01.403098 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-7c124e93-9a00-459b-ad8b-b86faa907ca4 None None] MFA Rules not processed for user `1eae540242d94c10b0ad33960a855f08`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:37:01.403305 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-1fbadde0-a1f6-486b-ba86-f074c665e598 None None] MFA Rules not processed for user `1eae540242d94c10b0ad33960a855f08`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:37:01.404311 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-1fbadde0-a1f6-486b-ba86-f074c665e598 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:37:01.404422 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-7c124e93-9a00-459b-ad8b-b86faa907ca4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:37:01.412062 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 785/1556] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 25 05:37:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:37:01.412628 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 772/1557] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 25 05:37:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:37:01.413305 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40467ec6-6dd0-473d-8056-cf2d02725ba1 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:37:01.413592 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40467ec6-6dd0-473d-8056-cf2d02725ba1 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:37:01.413872 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40467ec6-6dd0-473d-8056-cf2d02725ba1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:37:01.414053 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c124e93-9a00-459b-ad8b-b86faa907ca4 None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:37:01.426253 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c124e93-9a00-459b-ad8b-b86faa907ca4 None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:37:01.430887 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-804d6899-c7b1-43dd-aff2-9f2b3e37d33a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:37:01.431535 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804d6899-c7b1-43dd-aff2-9f2b3e37d33a admin admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:37:01.431818 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804d6899-c7b1-43dd-aff2-9f2b3e37d33a admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:37:01.432096 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804d6899-c7b1-43dd-aff2-9f2b3e37d33a admin admin] PATH_INFO: `/v3/roles` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:37:01.432898 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-40467ec6-6dd0-473d-8056-cf2d02725ba1 None None] MFA Rules not processed for user `1eae540242d94c10b0ad33960a855f08`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:37:01.433027 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-804d6899-c7b1-43dd-aff2-9f2b3e37d33a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:37:01.434071 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-40467ec6-6dd0-473d-8056-cf2d02725ba1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:37:01.434764 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-804d6899-c7b1-43dd-aff2-9f2b3e37d33a admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:37:01.442189 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 786/1558] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 25 05:37:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:37:01.443061 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40467ec6-6dd0-473d-8056-cf2d02725ba1 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:37:01.449138 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 773/1559] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jul 25 05:37:01 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 05:37:01.449138 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-804d6899-c7b1-43dd-aff2-9f2b3e37d33a admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:37:01.453374 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40467ec6-6dd0-473d-8056-cf2d02725ba1 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:37:01.458165 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc554125-8186-47a7-8a11-47c26a3af513 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:37:01.458820 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc554125-8186-47a7-8a11-47c26a3af513 admin admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:37:01.458943 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-804d6899-c7b1-43dd-aff2-9f2b3e37d33a admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:37:01.459105 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc554125-8186-47a7-8a11-47c26a3af513 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:37:01.459435 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc554125-8186-47a7-8a11-47c26a3af513 admin admin] PATH_INFO: `/v3/domains` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:37:01.460134 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc554125-8186-47a7-8a11-47c26a3af513 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:37:01.460443 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc554125-8186-47a7-8a11-47c26a3af513 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:37:01.462063 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc554125-8186-47a7-8a11-47c26a3af513 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:37:01.463466 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e77fb585-837b-428f-afee-73f4fc608103 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:37:01.464730 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77fb585-837b-428f-afee-73f4fc608103 admin admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:37:01.465062 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77fb585-837b-428f-afee-73f4fc608103 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:37:01.465357 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77fb585-837b-428f-afee-73f4fc608103 admin admin] PATH_INFO: `/v3/domains` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:37:01.466081 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e77fb585-837b-428f-afee-73f4fc608103 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:37:01.466397 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e77fb585-837b-428f-afee-73f4fc608103 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:37:01.467977 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e77fb585-837b-428f-afee-73f4fc608103 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:37:01.474670 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 787/1560] 10.222.0.33 () {58 vars in 1152 bytes} [Mon Jul 25 05:37:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:37:01.475458 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc554125-8186-47a7-8a11-47c26a3af513 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:37:01.483276 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 774/1561] 10.222.0.33 () {58 vars in 1152 bytes} [Mon Jul 25 05:37:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:37:01.484899 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ecd2f9-c025-432f-92de-3abd01a7f6cc None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:37:01.485188 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ecd2f9-c025-432f-92de-3abd01a7f6cc None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:37:01.485463 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ecd2f9-c025-432f-92de-3abd01a7f6cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:37:01.491237 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc554125-8186-47a7-8a11-47c26a3af513 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:37:01.496558 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d30cfeb-5d52-4b38-8f45-26b059db0dbf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:37:01.497288 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d30cfeb-5d52-4b38-8f45-26b059db0dbf admin admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:37:01.497585 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d30cfeb-5d52-4b38-8f45-26b059db0dbf admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:37:01.498009 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d30cfeb-5d52-4b38-8f45-26b059db0dbf admin admin] PATH_INFO: `/v3/domains` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:37:01.498894 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d30cfeb-5d52-4b38-8f45-26b059db0dbf admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:37:01.499211 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d30cfeb-5d52-4b38-8f45-26b059db0dbf admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:37:01.500843 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d30cfeb-5d52-4b38-8f45-26b059db0dbf admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:37:01.503016 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-17ecd2f9-c025-432f-92de-3abd01a7f6cc None None] MFA Rules not processed for user `1eae540242d94c10b0ad33960a855f08`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:37:01.504214 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-17ecd2f9-c025-432f-92de-3abd01a7f6cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:37:01.512232 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 775/1562] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 25 05:37:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:37:01.513496 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 788/1563] 10.222.0.33 () {58 vars in 1152 bytes} [Mon Jul 25 05:37:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:37:01.513805 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ecd2f9-c025-432f-92de-3abd01a7f6cc None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:37:01.514293 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d30cfeb-5d52-4b38-8f45-26b059db0dbf admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:37:01.524162 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d30cfeb-5d52-4b38-8f45-26b059db0dbf admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:37:01.525438 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ecd2f9-c025-432f-92de-3abd01a7f6cc None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:37:01.529633 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e5005d6-054c-4ab9-a018-fe9f6a88bf7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:37:01.530280 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5005d6-054c-4ab9-a018-fe9f6a88bf7e admin admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:37:01.530692 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5005d6-054c-4ab9-a018-fe9f6a88bf7e admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:37:01.530984 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c4bc23-690c-47fb-86de-50a8e4c8c7fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:37:01.531109 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5005d6-054c-4ab9-a018-fe9f6a88bf7e admin admin] PATH_INFO: `/v3/domains` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:37:01.531624 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c4bc23-690c-47fb-86de-50a8e4c8c7fc admin admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:37:01.531749 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e5005d6-054c-4ab9-a018-fe9f6a88bf7e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:37:01.531908 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c4bc23-690c-47fb-86de-50a8e4c8c7fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:37:01.532038 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e5005d6-054c-4ab9-a018-fe9f6a88bf7e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:37:01.532229 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c4bc23-690c-47fb-86de-50a8e4c8c7fc admin admin] PATH_INFO: `/v3/domains` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:37:01.533068 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00c4bc23-690c-47fb-86de-50a8e4c8c7fc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:37:01.533377 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00c4bc23-690c-47fb-86de-50a8e4c8c7fc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:37:01.533634 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e5005d6-054c-4ab9-a018-fe9f6a88bf7e admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:37:01.534945 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00c4bc23-690c-47fb-86de-50a8e4c8c7fc admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:37:01.546082 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 789/1564] 10.222.0.33 () {58 vars in 1152 bytes} [Mon Jul 25 05:37:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:37:01.546881 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e5005d6-054c-4ab9-a018-fe9f6a88bf7e admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:37:01.547508 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 776/1565] 10.222.0.33 () {58 vars in 1152 bytes} [Mon Jul 25 05:37:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:37:01.548306 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c4bc23-690c-47fb-86de-50a8e4c8c7fc admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:37:01.562247 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c4bc23-690c-47fb-86de-50a8e4c8c7fc admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:37:01.562787 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e5005d6-054c-4ab9-a018-fe9f6a88bf7e admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:37:01.566931 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d3d3998-a5aa-48cb-a759-13e535c2ac96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:37:01.567377 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aba6f06-85a8-4fe2-8d19-5f76328f3db0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:37:01.567817 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3d3998-a5aa-48cb-a759-13e535c2ac96 admin admin] REQUEST_METHOD: `PUT` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:37:01.568026 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aba6f06-85a8-4fe2-8d19-5f76328f3db0 admin admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:37:01.568133 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3d3998-a5aa-48cb-a759-13e535c2ac96 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:37:01.568313 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aba6f06-85a8-4fe2-8d19-5f76328f3db0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:37:01.568412 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3d3998-a5aa-48cb-a759-13e535c2ac96 admin admin] PATH_INFO: `/v3/projects/c782b01e8e484fe8bb710aeddc0192d1/users/37c1bb8500f74013b073febfc0d96217/roles/5074e74e35f54d4b9bb415c34999a82a` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:37:01.568592 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aba6f06-85a8-4fe2-8d19-5f76328f3db0 admin admin] PATH_INFO: `/v3/domains` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:37:01.569444 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aba6f06-85a8-4fe2-8d19-5f76328f3db0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:37:01.569756 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aba6f06-85a8-4fe2-8d19-5f76328f3db0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:37:01.571348 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aba6f06-85a8-4fe2-8d19-5f76328f3db0 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:37:01.583919 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d3d3998-a5aa-48cb-a759-13e535c2ac96 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c782b01e8e484fe8bb710aeddc0192d1, user_id=37c1bb8500f74013b073febfc0d96217, role_id=5074e74e35f54d4b9bb415c34999a82a)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:37:01.584654 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 790/1566] 10.222.0.33 () {58 vars in 1152 bytes} [Mon Jul 25 05:37:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:37:01.584985 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: WARNING py.warnings [None req-0d3d3998-a5aa-48cb-a759-13e535c2ac96 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 25 05:37:01.584985 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: warnings.warn(msg) Jul 25 05:37:01.584985 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: Jul 25 05:37:01.585485 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aba6f06-85a8-4fe2-8d19-5f76328f3db0 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:37:01.586917 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d3d3998-a5aa-48cb-a759-13e535c2ac96 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:37:01.595661 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aba6f06-85a8-4fe2-8d19-5f76328f3db0 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:37:01.599548 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:37:01.600890 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7900561-2d5f-4a5e-9f7d-f7c839c5beb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:37:01.601521 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7900561-2d5f-4a5e-9f7d-f7c839c5beb4 admin admin] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:37:01.601852 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7900561-2d5f-4a5e-9f7d-f7c839c5beb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:37:01.602165 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7900561-2d5f-4a5e-9f7d-f7c839c5beb4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:37:01.602851 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7900561-2d5f-4a5e-9f7d-f7c839c5beb4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:37:01.605355 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 777/1567] 10.222.0.33 () {60 vars in 1482 bytes} [Mon Jul 25 05:37:01 2022] PUT /identity/v3/projects/c782b01e8e484fe8bb710aeddc0192d1/users/37c1bb8500f74013b073febfc0d96217/roles/5074e74e35f54d4b9bb415c34999a82a => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 05:37:01.605903 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7900561-2d5f-4a5e-9f7d-f7c839c5beb4 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:37:01.606559 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d3d3998-a5aa-48cb-a759-13e535c2ac96 admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:37:01.639153 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 791/1568] 10.222.0.33 () {60 vars in 1152 bytes} [Mon Jul 25 05:37:01 2022] POST /identity/v3/projects => generated 371 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 05:37:01.639153 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7900561-2d5f-4a5e-9f7d-f7c839c5beb4 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:37:01.657562 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:37:01.675457 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d3d3998-a5aa-48cb-a759-13e535c2ac96 admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:37:01.700803 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7900561-2d5f-4a5e-9f7d-f7c839c5beb4 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:37:01.705713 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-969cd288-042c-485d-883e-a4cafd31c69e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:37:01.706450 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969cd288-042c-485d-883e-a4cafd31c69e admin admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:37:01.706748 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969cd288-042c-485d-883e-a4cafd31c69e admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:37:01.707032 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969cd288-042c-485d-883e-a4cafd31c69e admin admin] PATH_INFO: `/v3/domains` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:37:01.707771 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-969cd288-042c-485d-883e-a4cafd31c69e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:37:01.708084 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-969cd288-042c-485d-883e-a4cafd31c69e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:37:01.709730 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-969cd288-042c-485d-883e-a4cafd31c69e admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:37:01.722644 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 792/1569] 10.222.0.33 () {58 vars in 1152 bytes} [Mon Jul 25 05:37:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:37:01.723367 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-969cd288-042c-485d-883e-a4cafd31c69e admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:37:01.733576 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-969cd288-042c-485d-883e-a4cafd31c69e admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:37:01.733790 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b315cc73-11ed-4eb3-878b-5d021d2d31c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:37:01.734404 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b315cc73-11ed-4eb3-878b-5d021d2d31c2 admin admin] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:37:01.734689 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b315cc73-11ed-4eb3-878b-5d021d2d31c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:37:01.735000 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b315cc73-11ed-4eb3-878b-5d021d2d31c2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:37:01.735793 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b315cc73-11ed-4eb3-878b-5d021d2d31c2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:37:01.737780 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b315cc73-11ed-4eb3-878b-5d021d2d31c2 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:37:01.738241 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8985157c-241b-4732-aeef-7bdceac420ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:37:01.739045 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8985157c-241b-4732-aeef-7bdceac420ec admin admin] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:37:01.739320 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8985157c-241b-4732-aeef-7bdceac420ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:37:01.739665 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8985157c-241b-4732-aeef-7bdceac420ec admin admin] PATH_INFO: `/v3/projects` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:37:01.740417 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8985157c-241b-4732-aeef-7bdceac420ec admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:37:01.742399 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8985157c-241b-4732-aeef-7bdceac420ec admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:37:01.766964 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 778/1570] 10.222.0.33 () {60 vars in 1152 bytes} [Mon Jul 25 05:37:01 2022] POST /identity/v3/projects => generated 405 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 05:37:01.766964 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b315cc73-11ed-4eb3-878b-5d021d2d31c2 admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:37:01.769271 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 793/1571] 10.222.0.33 () {60 vars in 1152 bytes} [Mon Jul 25 05:37:01 2022] POST /identity/v3/projects => generated 383 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 05:37:01.770019 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8985157c-241b-4732-aeef-7bdceac420ec admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:37:01.834256 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b315cc73-11ed-4eb3-878b-5d021d2d31c2 admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:37:01.838918 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1acf988-4daf-4db1-9a50-2345199a1473 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:37:01.839838 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8985157c-241b-4732-aeef-7bdceac420ec admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:37:01.840308 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1acf988-4daf-4db1-9a50-2345199a1473 admin admin] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:37:01.840414 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1acf988-4daf-4db1-9a50-2345199a1473 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:37:01.840637 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1acf988-4daf-4db1-9a50-2345199a1473 admin admin] PATH_INFO: `/v3/projects` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:37:01.841386 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1acf988-4daf-4db1-9a50-2345199a1473 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:37:01.843590 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1acf988-4daf-4db1-9a50-2345199a1473 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:37:01.844870 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4087549-fc7e-4eee-ad17-4cc9a1359fe2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:37:01.845548 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4087549-fc7e-4eee-ad17-4cc9a1359fe2 admin admin] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:37:01.845818 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4087549-fc7e-4eee-ad17-4cc9a1359fe2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:37:01.846095 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4087549-fc7e-4eee-ad17-4cc9a1359fe2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:37:01.846785 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4087549-fc7e-4eee-ad17-4cc9a1359fe2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:37:01.848676 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4087549-fc7e-4eee-ad17-4cc9a1359fe2 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:37:01.872570 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 779/1572] 10.222.0.33 () {60 vars in 1152 bytes} [Mon Jul 25 05:37:01 2022] POST /identity/v3/projects => generated 405 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 05:37:01.873679 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1acf988-4daf-4db1-9a50-2345199a1473 admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:37:01.876847 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 794/1573] 10.222.0.33 () {60 vars in 1152 bytes} [Mon Jul 25 05:37:01 2022] POST /identity/v3/projects => generated 379 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 05:37:01.878258 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f624567-2e8a-4a08-b146-ff7440c4446e None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:37:01.878484 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f624567-2e8a-4a08-b146-ff7440c4446e None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:37:01.878712 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f624567-2e8a-4a08-b146-ff7440c4446e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:37:01.893151 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:37:01.895980 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-4f624567-2e8a-4a08-b146-ff7440c4446e None None] MFA Rules not processed for user `1eae540242d94c10b0ad33960a855f08`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:37:01.897249 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-4f624567-2e8a-4a08-b146-ff7440c4446e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:37:01.939954 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1acf988-4daf-4db1-9a50-2345199a1473 admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:37:01.957296 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 795/1574] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 25 05:37:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:37:01.960527 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f624567-2e8a-4a08-b146-ff7440c4446e None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:37:01.968310 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f624567-2e8a-4a08-b146-ff7440c4446e None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:37:01.972988 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc96ee4-8b05-48d7-8f45-0e39cf029415 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:37:01.973660 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc96ee4-8b05-48d7-8f45-0e39cf029415 admin admin] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:37:01.973953 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc96ee4-8b05-48d7-8f45-0e39cf029415 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:37:01.974235 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc96ee4-8b05-48d7-8f45-0e39cf029415 admin admin] PATH_INFO: `/v3/users` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:37:01.974965 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbc96ee4-8b05-48d7-8f45-0e39cf029415 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:37:01.975986 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: WARNING py.warnings [None req-bbc96ee4-8b05-48d7-8f45-0e39cf029415 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 25 05:37:01.975986 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: warnings.warn(msg) Jul 25 05:37:01.975986 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: Jul 25 05:37:01.977224 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbc96ee4-8b05-48d7-8f45-0e39cf029415 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:37:01.994391 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:37:02.003463 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 796/1575] 10.222.0.33 () {60 vars in 1143 bytes} [Mon Jul 25 05:37:01 2022] POST /identity/v3/users => generated 286 bytes in 42 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 05:37:02.003463 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc96ee4-8b05-48d7-8f45-0e39cf029415 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:37:02.003802 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-195d1aaa-a877-4006-b325-5499910a6069 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:37:02.003802 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195d1aaa-a877-4006-b325-5499910a6069 admin admin] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:37:02.003802 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195d1aaa-a877-4006-b325-5499910a6069 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:37:02.003802 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195d1aaa-a877-4006-b325-5499910a6069 admin admin] PATH_INFO: `/v3/projects` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:37:02.004135 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-195d1aaa-a877-4006-b325-5499910a6069 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:37:02.005388 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-195d1aaa-a877-4006-b325-5499910a6069 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:37:02.014350 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc96ee4-8b05-48d7-8f45-0e39cf029415 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:37:02.019060 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd6affd-8aae-4453-a6d7-9870a9a3727c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:37:02.019711 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd6affd-8aae-4453-a6d7-9870a9a3727c admin admin] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:37:02.020001 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd6affd-8aae-4453-a6d7-9870a9a3727c admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:37:02.020280 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd6affd-8aae-4453-a6d7-9870a9a3727c admin admin] PATH_INFO: `/v3/projects` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:37:02.021021 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fd6affd-8aae-4453-a6d7-9870a9a3727c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:37:02.023074 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fd6affd-8aae-4453-a6d7-9870a9a3727c admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:37:02.031794 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 780/1576] 10.222.0.33 () {60 vars in 1152 bytes} [Mon Jul 25 05:37:01 2022] POST /identity/v3/projects => generated 363 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 05:37:02.032547 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-195d1aaa-a877-4006-b325-5499910a6069 admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:37:02.051654 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 797/1577] 10.222.0.33 () {60 vars in 1152 bytes} [Mon Jul 25 05:37:02 2022] POST /identity/v3/projects => generated 385 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 05:37:02.052904 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd6affd-8aae-4453-a6d7-9870a9a3727c admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:37:02.097891 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-195d1aaa-a877-4006-b325-5499910a6069 admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:37:02.116361 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd6affd-8aae-4453-a6d7-9870a9a3727c admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:37:02.121678 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ace125-8ad5-4942-b453-0a104476b8fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:37:02.122406 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ace125-8ad5-4942-b453-0a104476b8fb admin admin] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:37:02.122695 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ace125-8ad5-4942-b453-0a104476b8fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:37:02.122968 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ace125-8ad5-4942-b453-0a104476b8fb admin admin] PATH_INFO: `/v3/users` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:37:02.123652 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64ace125-8ad5-4942-b453-0a104476b8fb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:37:02.125590 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64ace125-8ad5-4942-b453-0a104476b8fb admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:37:02.145624 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 798/1578] 10.222.0.33 () {60 vars in 1143 bytes} [Mon Jul 25 05:37:02 2022] POST /identity/v3/users => generated 292 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 05:37:02.146671 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ace125-8ad5-4942-b453-0a104476b8fb admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:37:02.149167 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:37:02.156870 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92632e3d-e943-4af4-b701-ee6c3eff0c16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:37:02.157767 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92632e3d-e943-4af4-b701-ee6c3eff0c16 admin admin] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:37:02.158234 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92632e3d-e943-4af4-b701-ee6c3eff0c16 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:37:02.158348 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ace125-8ad5-4942-b453-0a104476b8fb admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:37:02.158514 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92632e3d-e943-4af4-b701-ee6c3eff0c16 admin admin] PATH_INFO: `/v3/users` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:37:02.159215 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92632e3d-e943-4af4-b701-ee6c3eff0c16 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:37:02.161415 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92632e3d-e943-4af4-b701-ee6c3eff0c16 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:37:02.163154 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f4dec4-85af-4b8c-afb4-937c2b161dbf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:37:02.163823 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f4dec4-85af-4b8c-afb4-937c2b161dbf admin admin] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:37:02.164113 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f4dec4-85af-4b8c-afb4-937c2b161dbf admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:37:02.164392 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f4dec4-85af-4b8c-afb4-937c2b161dbf admin admin] PATH_INFO: `/v3/users` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:37:02.165141 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24f4dec4-85af-4b8c-afb4-937c2b161dbf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:37:02.167052 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24f4dec4-85af-4b8c-afb4-937c2b161dbf admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:37:02.183107 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 781/1579] 10.222.0.33 () {60 vars in 1143 bytes} [Mon Jul 25 05:37:02 2022] POST /identity/v3/users => generated 303 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 05:37:02.186868 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92632e3d-e943-4af4-b701-ee6c3eff0c16 admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:37:02.187790 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 799/1580] 10.222.0.33 () {60 vars in 1143 bytes} [Mon Jul 25 05:37:02 2022] POST /identity/v3/users => generated 303 bytes in 41 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 05:37:02.187790 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f4dec4-85af-4b8c-afb4-937c2b161dbf admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:37:02.190789 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-24f4dec4-85af-4b8c-afb4-937c2b161dbf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:37:02.196354 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92632e3d-e943-4af4-b701-ee6c3eff0c16 admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:37:02.201090 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-860789c7-6ace-4701-959c-29f23cff1b1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:37:02.201740 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860789c7-6ace-4701-959c-29f23cff1b1d admin admin] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:37:02.202026 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860789c7-6ace-4701-959c-29f23cff1b1d admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:37:02.202300 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860789c7-6ace-4701-959c-29f23cff1b1d admin admin] PATH_INFO: `/v3/users` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:37:02.202988 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-860789c7-6ace-4701-959c-29f23cff1b1d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:37:02.205274 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-860789c7-6ace-4701-959c-29f23cff1b1d admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:37:02.230370 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 782/1581] 10.222.0.33 () {60 vars in 1143 bytes} [Mon Jul 25 05:37:02 2022] POST /identity/v3/users => generated 290 bytes in 40 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 05:37:02.230370 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-860789c7-6ace-4701-959c-29f23cff1b1d admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:37:02.238464 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-860789c7-6ace-4701-959c-29f23cff1b1d admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:37:02.243234 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb9b03c-542e-4daa-9491-7bd580ccb4d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:37:02.243923 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb9b03c-542e-4daa-9491-7bd580ccb4d7 admin admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:37:02.244304 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb9b03c-542e-4daa-9491-7bd580ccb4d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:37:02.244589 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb9b03c-542e-4daa-9491-7bd580ccb4d7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:37:02.245496 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fb9b03c-542e-4daa-9491-7bd580ccb4d7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:37:02.247309 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fb9b03c-542e-4daa-9491-7bd580ccb4d7 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:37:02.251753 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f4dec4-85af-4b8c-afb4-937c2b161dbf admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:37:02.255743 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc92e0f-0d73-46c4-9a21-a7bda1700bfd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b2931200f99e43d79a50a9f6e02738e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92afba266cce4d1bad4134c351e74be3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:37:02.256389 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc92e0f-0d73-46c4-9a21-a7bda1700bfd service neutron] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:37:02.256690 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc92e0f-0d73-46c4-9a21-a7bda1700bfd service neutron] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:37:02.256984 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc92e0f-0d73-46c4-9a21-a7bda1700bfd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:37:02.257884 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 783/1582] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jul 25 05:37:02 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 05:37:02.258648 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb9b03c-542e-4daa-9491-7bd580ccb4d7 admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:37:02.268130 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acc92e0f-0d73-46c4-9a21-a7bda1700bfd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:37:02.269584 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acc92e0f-0d73-46c4-9a21-a7bda1700bfd service neutron] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:37:02.270908 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb9b03c-542e-4daa-9491-7bd580ccb4d7 admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:37:02.275502 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe73f359-2f2f-4013-b4fb-6aac8ce8949f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:37:02.276145 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe73f359-2f2f-4013-b4fb-6aac8ce8949f admin admin] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:37:02.276967 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe73f359-2f2f-4013-b4fb-6aac8ce8949f admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:37:02.277250 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe73f359-2f2f-4013-b4fb-6aac8ce8949f admin admin] PATH_INFO: `/v3/users` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:37:02.277928 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe73f359-2f2f-4013-b4fb-6aac8ce8949f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:37:02.279349 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 800/1583] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jul 25 05:37:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 05:37:02.279829 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe73f359-2f2f-4013-b4fb-6aac8ce8949f admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:37:02.280112 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc92e0f-0d73-46c4-9a21-a7bda1700bfd service neutron] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:37:02.289154 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc92e0f-0d73-46c4-9a21-a7bda1700bfd service neutron] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:37:02.295328 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e59ffdb-801c-403f-afab-ac00dc0151a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:37:02.295995 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e59ffdb-801c-403f-afab-ac00dc0151a3 admin admin] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:37:02.296429 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e59ffdb-801c-403f-afab-ac00dc0151a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:37:02.296697 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e59ffdb-801c-403f-afab-ac00dc0151a3 admin admin] PATH_INFO: `/v3/users` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:37:02.297432 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e59ffdb-801c-403f-afab-ac00dc0151a3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:37:02.299619 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e59ffdb-801c-403f-afab-ac00dc0151a3 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:37:02.308581 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 784/1584] 10.222.0.33 () {60 vars in 1143 bytes} [Mon Jul 25 05:37:02 2022] POST /identity/v3/users => generated 282 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 05:37:02.309306 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe73f359-2f2f-4013-b4fb-6aac8ce8949f admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:37:02.319844 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe73f359-2f2f-4013-b4fb-6aac8ce8949f admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:37:02.325002 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 801/1585] 10.222.0.33 () {60 vars in 1143 bytes} [Mon Jul 25 05:37:02 2022] POST /identity/v3/users => generated 293 bytes in 41 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 05:37:02.325002 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e59ffdb-801c-403f-afab-ac00dc0151a3 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:37:02.329265 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53913874-7b0e-48f9-9384-982d8e9df58f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:37:02.330002 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53913874-7b0e-48f9-9384-982d8e9df58f admin admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:37:02.330296 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53913874-7b0e-48f9-9384-982d8e9df58f admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:37:02.330627 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53913874-7b0e-48f9-9384-982d8e9df58f admin admin] PATH_INFO: `/v3/roles` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:37:02.331507 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53913874-7b0e-48f9-9384-982d8e9df58f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:37:02.332921 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e59ffdb-801c-403f-afab-ac00dc0151a3 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:37:02.333201 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53913874-7b0e-48f9-9384-982d8e9df58f admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:37:02.338326 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f156ae68-abd6-4fef-b206-7a6e30b9049c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:37:02.339706 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f156ae68-abd6-4fef-b206-7a6e30b9049c admin admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:37:02.340134 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f156ae68-abd6-4fef-b206-7a6e30b9049c admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:37:02.340414 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f156ae68-abd6-4fef-b206-7a6e30b9049c admin admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:37:02.341212 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f156ae68-abd6-4fef-b206-7a6e30b9049c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:37:02.342808 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f156ae68-abd6-4fef-b206-7a6e30b9049c admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:37:02.345998 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 785/1586] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jul 25 05:37:02 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 05:37:02.347105 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53913874-7b0e-48f9-9384-982d8e9df58f admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:37:02.354646 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 802/1587] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jul 25 05:37:02 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 05:37:02.355889 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f156ae68-abd6-4fef-b206-7a6e30b9049c admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:37:02.361496 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53913874-7b0e-48f9-9384-982d8e9df58f admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:37:02.367332 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae3226c4-38ed-4cc2-8306-0104743e010a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:37:02.367908 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f156ae68-abd6-4fef-b206-7a6e30b9049c admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:37:02.368055 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3226c4-38ed-4cc2-8306-0104743e010a admin admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:37:02.368284 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3226c4-38ed-4cc2-8306-0104743e010a admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:37:02.368566 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3226c4-38ed-4cc2-8306-0104743e010a admin admin] PATH_INFO: `/v3/roles` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:37:02.369339 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae3226c4-38ed-4cc2-8306-0104743e010a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:37:02.370971 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae3226c4-38ed-4cc2-8306-0104743e010a admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:37:02.373931 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d997e5-ce4a-4874-8b47-b2b9b965fd8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:37:02.374217 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d997e5-ce4a-4874-8b47-b2b9b965fd8e admin admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:37:02.374605 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d997e5-ce4a-4874-8b47-b2b9b965fd8e admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:37:02.374892 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d997e5-ce4a-4874-8b47-b2b9b965fd8e admin admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:37:02.375801 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1d997e5-ce4a-4874-8b47-b2b9b965fd8e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:37:02.377442 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1d997e5-ce4a-4874-8b47-b2b9b965fd8e admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:37:02.388015 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 786/1588] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jul 25 05:37:02 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 05:37:02.388015 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae3226c4-38ed-4cc2-8306-0104743e010a admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:37:02.395691 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 803/1589] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jul 25 05:37:02 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 05:37:02.395691 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d997e5-ce4a-4874-8b47-b2b9b965fd8e admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:37:02.397769 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae3226c4-38ed-4cc2-8306-0104743e010a admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:37:02.403083 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b95c40a-d771-4d70-abb1-915d4eea7f4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:37:02.403830 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b95c40a-d771-4d70-abb1-915d4eea7f4c admin admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:37:02.404186 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b95c40a-d771-4d70-abb1-915d4eea7f4c admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:37:02.404945 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b95c40a-d771-4d70-abb1-915d4eea7f4c admin admin] PATH_INFO: `/v3/roles` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:37:02.405446 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b95c40a-d771-4d70-abb1-915d4eea7f4c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:37:02.406737 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d997e5-ce4a-4874-8b47-b2b9b965fd8e admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:37:02.407229 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b95c40a-d771-4d70-abb1-915d4eea7f4c admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:37:02.411641 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbbf9648-4a78-42c8-9938-c3441522a201 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:37:02.412304 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbbf9648-4a78-42c8-9938-c3441522a201 admin admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:37:02.412588 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbbf9648-4a78-42c8-9938-c3441522a201 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:37:02.412896 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbbf9648-4a78-42c8-9938-c3441522a201 admin admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:37:02.413629 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbbf9648-4a78-42c8-9938-c3441522a201 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:37:02.415249 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbbf9648-4a78-42c8-9938-c3441522a201 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:37:02.419792 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 787/1590] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jul 25 05:37:02 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 05:37:02.424608 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b95c40a-d771-4d70-abb1-915d4eea7f4c admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:37:02.430221 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 804/1591] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jul 25 05:37:02 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 05:37:02.430221 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbbf9648-4a78-42c8-9938-c3441522a201 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:37:02.436410 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=90,keystone:INSERT=10 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:37:02.436996 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b95c40a-d771-4d70-abb1-915d4eea7f4c admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:37:02.442804 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbbf9648-4a78-42c8-9938-c3441522a201 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:37:02.444096 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b457d52-6720-422c-8c11-cd000240fca7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:37:02.445074 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b457d52-6720-422c-8c11-cd000240fca7 admin admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:37:02.445465 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b457d52-6720-422c-8c11-cd000240fca7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:37:02.445642 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b457d52-6720-422c-8c11-cd000240fca7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:37:02.447569 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b457d52-6720-422c-8c11-cd000240fca7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:37:02.449814 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b457d52-6720-422c-8c11-cd000240fca7 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:37:02.451376 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb8fe6f3-4fd2-45ab-9eb1-350580f590ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:37:02.452168 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8fe6f3-4fd2-45ab-9eb1-350580f590ed admin admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:37:02.452414 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8fe6f3-4fd2-45ab-9eb1-350580f590ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:37:02.453102 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8fe6f3-4fd2-45ab-9eb1-350580f590ed admin admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:37:02.454037 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb8fe6f3-4fd2-45ab-9eb1-350580f590ed admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:37:02.455770 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb8fe6f3-4fd2-45ab-9eb1-350580f590ed admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:37:02.465581 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 788/1592] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jul 25 05:37:02 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 05:37:02.465581 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b457d52-6720-422c-8c11-cd000240fca7 admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:37:02.470287 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 805/1593] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jul 25 05:37:02 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 05:37:02.470287 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb8fe6f3-4fd2-45ab-9eb1-350580f590ed admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:37:02.477604 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b457d52-6720-422c-8c11-cd000240fca7 admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:37:02.481490 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb8fe6f3-4fd2-45ab-9eb1-350580f590ed admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:37:02.483215 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf2c597b-c610-4b75-8f78-f3ad69def6f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:37:02.483886 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2c597b-c610-4b75-8f78-f3ad69def6f1 admin admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:37:02.484177 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2c597b-c610-4b75-8f78-f3ad69def6f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:37:02.484455 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2c597b-c610-4b75-8f78-f3ad69def6f1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:37:02.485227 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf2c597b-c610-4b75-8f78-f3ad69def6f1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:37:02.486760 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c15694f8-8279-4fac-b952-319ceb6641ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:37:02.486899 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf2c597b-c610-4b75-8f78-f3ad69def6f1 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:37:02.487592 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15694f8-8279-4fac-b952-319ceb6641ab admin admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:37:02.487987 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15694f8-8279-4fac-b952-319ceb6641ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:37:02.488269 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15694f8-8279-4fac-b952-319ceb6641ab admin admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:37:02.489069 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c15694f8-8279-4fac-b952-319ceb6641ab admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:37:02.490984 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c15694f8-8279-4fac-b952-319ceb6641ab admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:37:02.502602 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 789/1594] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jul 25 05:37:02 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 05:37:02.502602 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf2c597b-c610-4b75-8f78-f3ad69def6f1 admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:37:02.502831 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 806/1595] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jul 25 05:37:02 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 05:37:02.503276 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c15694f8-8279-4fac-b952-319ceb6641ab admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:37:02.515985 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c15694f8-8279-4fac-b952-319ceb6641ab admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:37:02.521044 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-932d0f61-c398-44f8-83be-6e2688d846a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:37:02.521894 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932d0f61-c398-44f8-83be-6e2688d846a6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:37:02.522181 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932d0f61-c398-44f8-83be-6e2688d846a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:37:02.522512 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932d0f61-c398-44f8-83be-6e2688d846a6 admin admin] PATH_INFO: `/v3/projects/129b404cb6484de8962c1aa49e34cca1/users/316fd0c7342946638423ecfc7a506dfb/roles/5074e74e35f54d4b9bb415c34999a82a` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:37:02.533218 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf2c597b-c610-4b75-8f78-f3ad69def6f1 admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:37:02.539137 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e818f0-c5ad-42ce-b62e-a04ab37b9b60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:37:02.539915 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e818f0-c5ad-42ce-b62e-a04ab37b9b60 admin admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:37:02.540208 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e818f0-c5ad-42ce-b62e-a04ab37b9b60 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:37:02.540439 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-932d0f61-c398-44f8-83be-6e2688d846a6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=129b404cb6484de8962c1aa49e34cca1, user_id=316fd0c7342946638423ecfc7a506dfb, role_id=5074e74e35f54d4b9bb415c34999a82a)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:37:02.540568 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e818f0-c5ad-42ce-b62e-a04ab37b9b60 admin admin] PATH_INFO: `/v3/roles` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:37:02.541654 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53e818f0-c5ad-42ce-b62e-a04ab37b9b60 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:37:02.542961 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53e818f0-c5ad-42ce-b62e-a04ab37b9b60 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:37:02.543674 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-932d0f61-c398-44f8-83be-6e2688d846a6 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:37:02.554456 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 790/1596] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jul 25 05:37:02 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 05:37:02.555429 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e818f0-c5ad-42ce-b62e-a04ab37b9b60 admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:37:02.561173 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 807/1597] 10.222.0.33 () {60 vars in 1482 bytes} [Mon Jul 25 05:37:02 2022] PUT /identity/v3/projects/129b404cb6484de8962c1aa49e34cca1/users/316fd0c7342946638423ecfc7a506dfb/roles/5074e74e35f54d4b9bb415c34999a82a => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 05:37:02.561998 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-932d0f61-c398-44f8-83be-6e2688d846a6 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:37:02.566356 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e818f0-c5ad-42ce-b62e-a04ab37b9b60 admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:37:02.647937 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c031a4e-800e-476f-8d41-80adf55f1547 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:37:02.648723 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c031a4e-800e-476f-8d41-80adf55f1547 admin admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:37:02.648945 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c031a4e-800e-476f-8d41-80adf55f1547 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:37:02.649206 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c031a4e-800e-476f-8d41-80adf55f1547 admin admin] PATH_INFO: `/v3/roles` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:37:02.650290 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c031a4e-800e-476f-8d41-80adf55f1547 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:37:02.653212 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c031a4e-800e-476f-8d41-80adf55f1547 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:37:02.655100 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-932d0f61-c398-44f8-83be-6e2688d846a6 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:37:02.660570 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0549c956-eb97-43dd-a4f2-d9d2e1e2fd4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:37:02.661303 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0549c956-eb97-43dd-a4f2-d9d2e1e2fd4b admin admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:37:02.661588 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0549c956-eb97-43dd-a4f2-d9d2e1e2fd4b admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:37:02.661864 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0549c956-eb97-43dd-a4f2-d9d2e1e2fd4b admin admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:37:02.663176 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0549c956-eb97-43dd-a4f2-d9d2e1e2fd4b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:37:02.664391 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0549c956-eb97-43dd-a4f2-d9d2e1e2fd4b admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:37:02.665402 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 791/1598] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jul 25 05:37:02 2022] GET /identity/v3/roles => generated 1420 bytes in 110 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 05:37:02.666315 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c031a4e-800e-476f-8d41-80adf55f1547 admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:37:02.680176 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 808/1599] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jul 25 05:37:02 2022] GET /identity/v3/roles => generated 1420 bytes in 115 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 05:37:02.680176 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0549c956-eb97-43dd-a4f2-d9d2e1e2fd4b admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:37:02.680496 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c031a4e-800e-476f-8d41-80adf55f1547 admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:37:02.684540 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a7f0091-7bf6-4540-8765-371d48c2b6fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:37:02.686076 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7f0091-7bf6-4540-8765-371d48c2b6fd admin admin] REQUEST_METHOD: `PUT` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:37:02.686432 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7f0091-7bf6-4540-8765-371d48c2b6fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:37:02.686998 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7f0091-7bf6-4540-8765-371d48c2b6fd admin admin] PATH_INFO: `/v3/projects/aa82c2fa3dea499a8b5f1e7d5fcd8acd/users/da3b4bd3f89842c7af66f9761ba87116/roles/5074e74e35f54d4b9bb415c34999a82a` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:37:02.689369 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0549c956-eb97-43dd-a4f2-d9d2e1e2fd4b admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:37:02.694740 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fbfa1bb-c248-499a-b3cc-556949bf19c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:37:02.695763 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fbfa1bb-c248-499a-b3cc-556949bf19c6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:37:02.696113 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fbfa1bb-c248-499a-b3cc-556949bf19c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:37:02.696455 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fbfa1bb-c248-499a-b3cc-556949bf19c6 admin admin] PATH_INFO: `/v3/projects/039d0c9e35dd4a74b7b08c982535155d/users/aec7125a032f42d19b106b79c9968d46/roles/5074e74e35f54d4b9bb415c34999a82a` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:37:02.713747 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a7f0091-7bf6-4540-8765-371d48c2b6fd admin admin] RBAC: Authorizing `identity:create_grant(project_id=aa82c2fa3dea499a8b5f1e7d5fcd8acd, user_id=da3b4bd3f89842c7af66f9761ba87116, role_id=5074e74e35f54d4b9bb415c34999a82a)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:37:02.717630 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a7f0091-7bf6-4540-8765-371d48c2b6fd admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:37:02.719161 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fbfa1bb-c248-499a-b3cc-556949bf19c6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=039d0c9e35dd4a74b7b08c982535155d, user_id=aec7125a032f42d19b106b79c9968d46, role_id=5074e74e35f54d4b9bb415c34999a82a)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:37:02.722638 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fbfa1bb-c248-499a-b3cc-556949bf19c6 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:37:02.735952 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 792/1600] 10.222.0.33 () {60 vars in 1482 bytes} [Mon Jul 25 05:37:02 2022] PUT /identity/v3/projects/aa82c2fa3dea499a8b5f1e7d5fcd8acd/users/da3b4bd3f89842c7af66f9761ba87116/roles/5074e74e35f54d4b9bb415c34999a82a => generated 0 bytes in 70 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 05:37:02.752083 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a7f0091-7bf6-4540-8765-371d48c2b6fd admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:37:02.752181 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 809/1601] 10.222.0.33 () {60 vars in 1482 bytes} [Mon Jul 25 05:37:02 2022] PUT /identity/v3/projects/039d0c9e35dd4a74b7b08c982535155d/users/aec7125a032f42d19b106b79c9968d46/roles/5074e74e35f54d4b9bb415c34999a82a => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 05:37:02.752181 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fbfa1bb-c248-499a-b3cc-556949bf19c6 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:37:02.752360 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=90,keystone:INSERT=10 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:37:02.812533 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a7f0091-7bf6-4540-8765-371d48c2b6fd admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:37:02.817830 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34d0ccf-0252-49c8-8444-bdd9a538d7f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:37:02.818754 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34d0ccf-0252-49c8-8444-bdd9a538d7f2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:37:02.819036 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34d0ccf-0252-49c8-8444-bdd9a538d7f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:37:02.819310 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34d0ccf-0252-49c8-8444-bdd9a538d7f2 admin admin] PATH_INFO: `/v3/projects/2854c008a6674dc1b63c6550d0f2108f/users/f2008c3f679441428a89babc69b76a7d/roles/5074e74e35f54d4b9bb415c34999a82a` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:37:02.819423 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fbfa1bb-c248-499a-b3cc-556949bf19c6 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:37:02.824884 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab027995-4902-4af3-8627-7f75ce99b79e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:37:02.826535 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab027995-4902-4af3-8627-7f75ce99b79e admin admin] REQUEST_METHOD: `PUT` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:37:02.826900 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab027995-4902-4af3-8627-7f75ce99b79e admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:37:02.827236 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab027995-4902-4af3-8627-7f75ce99b79e admin admin] PATH_INFO: `/v3/projects/0dc32f383ef54d94aeb44d0bc3885614/users/bf61ffac0e3143f6b44971158bd7a553/roles/5074e74e35f54d4b9bb415c34999a82a` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:37:02.836512 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f34d0ccf-0252-49c8-8444-bdd9a538d7f2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2854c008a6674dc1b63c6550d0f2108f, user_id=f2008c3f679441428a89babc69b76a7d, role_id=5074e74e35f54d4b9bb415c34999a82a)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:37:02.839300 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f34d0ccf-0252-49c8-8444-bdd9a538d7f2 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:37:02.846115 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab027995-4902-4af3-8627-7f75ce99b79e admin admin] RBAC: Authorizing `identity:create_grant(project_id=0dc32f383ef54d94aeb44d0bc3885614, user_id=bf61ffac0e3143f6b44971158bd7a553, role_id=5074e74e35f54d4b9bb415c34999a82a)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:37:02.848978 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab027995-4902-4af3-8627-7f75ce99b79e admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:37:02.855033 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 793/1602] 10.222.0.33 () {60 vars in 1482 bytes} [Mon Jul 25 05:37:02 2022] PUT /identity/v3/projects/2854c008a6674dc1b63c6550d0f2108f/users/f2008c3f679441428a89babc69b76a7d/roles/5074e74e35f54d4b9bb415c34999a82a => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 05:37:02.856499 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161a68e8-ada8-492c-9722-e53c72fcaca4 None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:37:02.856736 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161a68e8-ada8-492c-9722-e53c72fcaca4 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:37:02.857040 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161a68e8-ada8-492c-9722-e53c72fcaca4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:37:02.874101 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 810/1603] 10.222.0.33 () {60 vars in 1482 bytes} [Mon Jul 25 05:37:02 2022] PUT /identity/v3/projects/0dc32f383ef54d94aeb44d0bc3885614/users/bf61ffac0e3143f6b44971158bd7a553/roles/5074e74e35f54d4b9bb415c34999a82a => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 05:37:02.874771 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab027995-4902-4af3-8627-7f75ce99b79e admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:37:02.889802 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-161a68e8-ada8-492c-9722-e53c72fcaca4 None None] MFA Rules not processed for user `316fd0c7342946638423ecfc7a506dfb`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:37:02.891019 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-161a68e8-ada8-492c-9722-e53c72fcaca4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:37:02.892290 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:37:02.952390 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab027995-4902-4af3-8627-7f75ce99b79e admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:37:02.961255 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa6ae39e-5801-416a-a6a7-fffcc6502130 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:37:02.962742 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6ae39e-5801-416a-a6a7-fffcc6502130 admin admin] REQUEST_METHOD: `PUT` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:37:02.963033 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6ae39e-5801-416a-a6a7-fffcc6502130 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:37:02.963323 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6ae39e-5801-416a-a6a7-fffcc6502130 admin admin] PATH_INFO: `/v3/projects/5c72716a74ac49f58d962efcba3e9962/users/00d1cb0b04034ce882be46b34eda6618/roles/5074e74e35f54d4b9bb415c34999a82a` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:37:02.975226 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 794/1604] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 25 05:37:02 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:37:02.976015 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-161a68e8-ada8-492c-9722-e53c72fcaca4 None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:37:02.982572 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa6ae39e-5801-416a-a6a7-fffcc6502130 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5c72716a74ac49f58d962efcba3e9962, user_id=00d1cb0b04034ce882be46b34eda6618, role_id=5074e74e35f54d4b9bb415c34999a82a)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:37:02.985820 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa6ae39e-5801-416a-a6a7-fffcc6502130 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:37:02.989592 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-161a68e8-ada8-492c-9722-e53c72fcaca4 None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:37:02.999057 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe85fd7b-5e80-48f7-9f22-ffcedcf3d10f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:37:02.999991 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe85fd7b-5e80-48f7-9f22-ffcedcf3d10f admin admin] REQUEST_METHOD: `PUT` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:37:03.000817 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe85fd7b-5e80-48f7-9f22-ffcedcf3d10f admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:37:03.000817 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe85fd7b-5e80-48f7-9f22-ffcedcf3d10f admin admin] PATH_INFO: `/v3/projects/42066c73ac8f4e5c883204dfc66245c0/users/69f3276517de4f889dddd3180436ede9/roles/5074e74e35f54d4b9bb415c34999a82a` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:37:03.019024 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe85fd7b-5e80-48f7-9f22-ffcedcf3d10f admin admin] RBAC: Authorizing `identity:create_grant(project_id=42066c73ac8f4e5c883204dfc66245c0, user_id=69f3276517de4f889dddd3180436ede9, role_id=5074e74e35f54d4b9bb415c34999a82a)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:37:03.021798 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 811/1605] 10.222.0.33 () {60 vars in 1482 bytes} [Mon Jul 25 05:37:02 2022] PUT /identity/v3/projects/5c72716a74ac49f58d962efcba3e9962/users/00d1cb0b04034ce882be46b34eda6618/roles/5074e74e35f54d4b9bb415c34999a82a => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 05:37:03.022418 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe85fd7b-5e80-48f7-9f22-ffcedcf3d10f admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:37:03.035156 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78099db0-86a6-42c1-95db-38c339d15f4b None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:37:03.035598 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78099db0-86a6-42c1-95db-38c339d15f4b None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:37:03.035711 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78099db0-86a6-42c1-95db-38c339d15f4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:37:03.045772 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 795/1606] 10.222.0.33 () {60 vars in 1482 bytes} [Mon Jul 25 05:37:02 2022] PUT /identity/v3/projects/42066c73ac8f4e5c883204dfc66245c0/users/69f3276517de4f889dddd3180436ede9/roles/5074e74e35f54d4b9bb415c34999a82a => generated 0 bytes in 70 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 05:37:03.051041 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2161120-e019-4013-ac57-472be40244b5 None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:37:03.051454 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2161120-e019-4013-ac57-472be40244b5 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:37:03.051793 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2161120-e019-4013-ac57-472be40244b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:37:03.062118 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-78099db0-86a6-42c1-95db-38c339d15f4b None None] MFA Rules not processed for user `1eae540242d94c10b0ad33960a855f08`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:37:03.064089 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-78099db0-86a6-42c1-95db-38c339d15f4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:37:03.086964 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-b2161120-e019-4013-ac57-472be40244b5 None None] MFA Rules not processed for user `da3b4bd3f89842c7af66f9761ba87116`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:37:03.088236 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-b2161120-e019-4013-ac57-472be40244b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:37:03.148096 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 812/1607] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 25 05:37:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:37:03.150584 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d99a25-c528-4dc6-b98c-1e85a7ba1410 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:37:03.150880 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d99a25-c528-4dc6-b98c-1e85a7ba1410 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:37:03.161100 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:37:03.162582 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d99a25-c528-4dc6-b98c-1e85a7ba1410 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:37:03.194918 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 796/1608] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 25 05:37:03 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:37:03.201707 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4075693b-f1ff-41de-bdd7-dbfaafb4a104 None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:37:03.202243 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4075693b-f1ff-41de-bdd7-dbfaafb4a104 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:37:03.202705 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4075693b-f1ff-41de-bdd7-dbfaafb4a104 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:37:03.204920 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-37d99a25-c528-4dc6-b98c-1e85a7ba1410 None None] MFA Rules not processed for user `f2008c3f679441428a89babc69b76a7d`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:37:03.206377 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-37d99a25-c528-4dc6-b98c-1e85a7ba1410 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:37:03.225114 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-4075693b-f1ff-41de-bdd7-dbfaafb4a104 None None] MFA Rules not processed for user `1eae540242d94c10b0ad33960a855f08`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:37:03.227535 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-4075693b-f1ff-41de-bdd7-dbfaafb4a104 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:37:03.239136 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 797/1609] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 25 05:37:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:37:03.241066 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e825d3-f34f-49b6-9f33-c8f6e0c8b489 None None] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:37:03.241669 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e825d3-f34f-49b6-9f33-c8f6e0c8b489 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:37:03.242498 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e825d3-f34f-49b6-9f33-c8f6e0c8b489 None None] PATH_INFO: `/` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:37:03.246334 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 798/1610] 10.222.0.33 () {54 vars in 962 bytes} [Mon Jul 25 05:37:03 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:37:03.248098 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf38982-7a5c-4f94-8560-7b3fb61519b4 None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:37:03.248885 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf38982-7a5c-4f94-8560-7b3fb61519b4 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:37:03.249608 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf38982-7a5c-4f94-8560-7b3fb61519b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:37:03.285771 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-8cf38982-7a5c-4f94-8560-7b3fb61519b4 None None] MFA Rules not processed for user `00d1cb0b04034ce882be46b34eda6618`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:37:03.288109 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-8cf38982-7a5c-4f94-8560-7b3fb61519b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:37:03.306744 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 813/1611] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 25 05:37:03 2022] POST /identity/v3/auth/tokens => generated 3272 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:37:03.310710 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1700ee-e5dc-41d5-bad9-c88ba599a819 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:37:03.311013 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1700ee-e5dc-41d5-bad9-c88ba599a819 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:37:03.311304 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1700ee-e5dc-41d5-bad9-c88ba599a819 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:37:03.343655 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-dc1700ee-e5dc-41d5-bad9-c88ba599a819 None None] MFA Rules not processed for user `1eae540242d94c10b0ad33960a855f08`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:37:03.345277 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-dc1700ee-e5dc-41d5-bad9-c88ba599a819 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:37:03.358453 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 814/1612] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 25 05:37:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:37:03.359649 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc1700ee-e5dc-41d5-bad9-c88ba599a819 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:37:03.383260 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:37:03.385266 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 799/1613] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 25 05:37:03 2022] POST /identity/v3/auth/tokens => generated 3230 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:37:03.387200 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf38982-7a5c-4f94-8560-7b3fb61519b4 None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:37:03.439558 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc1700ee-e5dc-41d5-bad9-c88ba599a819 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:37:03.445198 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d12ae5f6-2738-46e9-a1d4-9fa0b166139e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b2931200f99e43d79a50a9f6e02738e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92afba266cce4d1bad4134c351e74be3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:37:03.447407 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12ae5f6-2738-46e9-a1d4-9fa0b166139e service neutron] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:37:03.447887 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12ae5f6-2738-46e9-a1d4-9fa0b166139e service neutron] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:37:03.448376 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12ae5f6-2738-46e9-a1d4-9fa0b166139e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:37:03.453042 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf38982-7a5c-4f94-8560-7b3fb61519b4 None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:37:03.457137 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-344b7219-b8be-4f39-89a6-29c67a2dca41 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b2931200f99e43d79a50a9f6e02738e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92afba266cce4d1bad4134c351e74be3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:37:03.458302 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344b7219-b8be-4f39-89a6-29c67a2dca41 service neutron] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:37:03.458302 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344b7219-b8be-4f39-89a6-29c67a2dca41 service neutron] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:37:03.458534 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344b7219-b8be-4f39-89a6-29c67a2dca41 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:37:03.461152 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d12ae5f6-2738-46e9-a1d4-9fa0b166139e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:37:03.463106 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d12ae5f6-2738-46e9-a1d4-9fa0b166139e service neutron] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:37:03.470176 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-344b7219-b8be-4f39-89a6-29c67a2dca41 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:37:03.471704 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-344b7219-b8be-4f39-89a6-29c67a2dca41 service neutron] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:37:03.478191 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 815/1614] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jul 25 05:37:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 05:37:03.480630 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d1a112-64f9-45a4-a55d-5721a6dcf0ce None None] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:37:03.481164 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d1a112-64f9-45a4-a55d-5721a6dcf0ce None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:37:03.481601 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d1a112-64f9-45a4-a55d-5721a6dcf0ce None None] PATH_INFO: `/` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:37:03.488897 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 816/1615] 10.222.0.33 () {54 vars in 962 bytes} [Mon Jul 25 05:37:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:37:03.490711 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd452057-945d-46ad-a2e6-0048634d3d0b None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:37:03.491767 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd452057-945d-46ad-a2e6-0048634d3d0b None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:37:03.492226 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd452057-945d-46ad-a2e6-0048634d3d0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:37:03.502381 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 800/1616] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jul 25 05:37:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 05:37:03.503190 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-344b7219-b8be-4f39-89a6-29c67a2dca41 service neutron] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:37:03.523848 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-344b7219-b8be-4f39-89a6-29c67a2dca41 service neutron] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:37:03.540035 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca0ae5ca-d72c-4d2e-a268-0f6cd8066d03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:37:03.540854 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0ae5ca-d72c-4d2e-a268-0f6cd8066d03 admin admin] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:37:03.541144 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0ae5ca-d72c-4d2e-a268-0f6cd8066d03 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:37:03.541475 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0ae5ca-d72c-4d2e-a268-0f6cd8066d03 admin admin] PATH_INFO: `/v3/projects` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:37:03.541753 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-cd452057-945d-46ad-a2e6-0048634d3d0b None None] MFA Rules not processed for user `1cd482a61e8a420db45bb532e5dae6e2`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:37:03.542328 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca0ae5ca-d72c-4d2e-a268-0f6cd8066d03 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:37:03.543547 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-cd452057-945d-46ad-a2e6-0048634d3d0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:37:03.547938 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca0ae5ca-d72c-4d2e-a268-0f6cd8066d03 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:37:03.606548 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 801/1617] 10.222.0.33 () {60 vars in 1152 bytes} [Mon Jul 25 05:37:03 2022] POST /identity/v3/projects => generated 403 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 05:37:03.607318 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca0ae5ca-d72c-4d2e-a268-0f6cd8066d03 admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:37:03.627729 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 817/1618] 10.222.0.33 () {58 vars in 1059 bytes} [Mon Jul 25 05:37:03 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:37:03.629000 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd452057-945d-46ad-a2e6-0048634d3d0b None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:37:03.657003 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd452057-945d-46ad-a2e6-0048634d3d0b None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:37:03.769165 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:37:03.771337 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb997ad-487f-4781-92ac-e4202579e3e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:37:03.773928 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb997ad-487f-4781-92ac-e4202579e3e8 admin admin] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:37:03.775804 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb997ad-487f-4781-92ac-e4202579e3e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:37:03.776232 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb997ad-487f-4781-92ac-e4202579e3e8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:37:03.778544 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfb997ad-487f-4781-92ac-e4202579e3e8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:37:03.780956 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca0ae5ca-d72c-4d2e-a268-0f6cd8066d03 admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:37:03.783640 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfb997ad-487f-4781-92ac-e4202579e3e8 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:37:03.792830 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-534cc65d-b1b7-48b3-b15b-8e99aa89f5d4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b2931200f99e43d79a50a9f6e02738e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92afba266cce4d1bad4134c351e74be3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:37:03.794240 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534cc65d-b1b7-48b3-b15b-8e99aa89f5d4 service neutron] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:37:03.794532 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534cc65d-b1b7-48b3-b15b-8e99aa89f5d4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:37:03.794818 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534cc65d-b1b7-48b3-b15b-8e99aa89f5d4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:37:03.819493 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-534cc65d-b1b7-48b3-b15b-8e99aa89f5d4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:37:03.821320 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-534cc65d-b1b7-48b3-b15b-8e99aa89f5d4 service neutron] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:37:03.836755 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 802/1619] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jul 25 05:37:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 230 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 05:37:03.838365 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a9abe3-a90a-4544-a26e-b43e19d257ca None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:37:03.842992 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a9abe3-a90a-4544-a26e-b43e19d257ca None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:37:03.842992 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a9abe3-a90a-4544-a26e-b43e19d257ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:37:03.844258 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 818/1620] 10.222.0.33 () {60 vars in 1152 bytes} [Mon Jul 25 05:37:03 2022] POST /identity/v3/projects => generated 363 bytes in 216 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 05:37:03.845460 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb997ad-487f-4781-92ac-e4202579e3e8 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:37:03.863354 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-73a9abe3-a90a-4544-a26e-b43e19d257ca None None] MFA Rules not processed for user `1cd482a61e8a420db45bb532e5dae6e2`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:37:03.864885 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-73a9abe3-a90a-4544-a26e-b43e19d257ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:37:03.899306 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:37:03.966086 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 803/1621] 10.222.0.33 () {58 vars in 1059 bytes} [Mon Jul 25 05:37:03 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:37:03.976961 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a9abe3-a90a-4544-a26e-b43e19d257ca None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:37:03.992398 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a9abe3-a90a-4544-a26e-b43e19d257ca None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:37:04.005212 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cac9ed0-17f1-4aa4-922e-04b8c32d9c40 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cd482a61e8a420db45bb532e5dae6e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92afba266cce4d1bad4134c351e74be3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:37:04.010792 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cac9ed0-17f1-4aa4-922e-04b8c32d9c40 service nova] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:37:04.011537 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cac9ed0-17f1-4aa4-922e-04b8c32d9c40 service nova] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:37:04.011825 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cac9ed0-17f1-4aa4-922e-04b8c32d9c40 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:37:04.020295 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb997ad-487f-4781-92ac-e4202579e3e8 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:37:04.027442 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba18b9d-6f8b-4577-944f-15d2cf7b0be8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:37:04.028336 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba18b9d-6f8b-4577-944f-15d2cf7b0be8 admin admin] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:37:04.032310 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba18b9d-6f8b-4577-944f-15d2cf7b0be8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:37:04.032310 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba18b9d-6f8b-4577-944f-15d2cf7b0be8 admin admin] PATH_INFO: `/v3/users` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:37:04.032310 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ba18b9d-6f8b-4577-944f-15d2cf7b0be8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:37:04.032310 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ba18b9d-6f8b-4577-944f-15d2cf7b0be8 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:37:04.069323 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 819/1622] 10.222.0.33 () {60 vars in 1143 bytes} [Mon Jul 25 05:37:03 2022] POST /identity/v3/users => generated 308 bytes in 224 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 05:37:04.078952 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba18b9d-6f8b-4577-944f-15d2cf7b0be8 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:37:04.083388 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cac9ed0-17f1-4aa4-922e-04b8c32d9c40 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:37:04.085011 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cac9ed0-17f1-4aa4-922e-04b8c32d9c40 service nova] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:37:04.097569 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba18b9d-6f8b-4577-944f-15d2cf7b0be8 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:37:04.097880 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 804/1623] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jul 25 05:37:03 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 05:37:04.097880 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cac9ed0-17f1-4aa4-922e-04b8c32d9c40 service nova] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:37:04.108626 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cac9ed0-17f1-4aa4-922e-04b8c32d9c40 service nova] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:37:04.114620 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc8a29a-29d8-467a-b6a1-53b420a3743c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cd482a61e8a420db45bb532e5dae6e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92afba266cce4d1bad4134c351e74be3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:37:04.115530 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc8a29a-29d8-467a-b6a1-53b420a3743c service nova] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:37:04.115813 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc8a29a-29d8-467a-b6a1-53b420a3743c service nova] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:37:04.116288 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc8a29a-29d8-467a-b6a1-53b420a3743c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:37:04.122271 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3168efb-4d8e-4ba8-b888-7aae6dd1dabf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:37:04.123370 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3168efb-4d8e-4ba8-b888-7aae6dd1dabf admin admin] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:37:04.126009 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3168efb-4d8e-4ba8-b888-7aae6dd1dabf admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:37:04.126698 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3168efb-4d8e-4ba8-b888-7aae6dd1dabf admin admin] PATH_INFO: `/v3/users` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:37:04.127419 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3168efb-4d8e-4ba8-b888-7aae6dd1dabf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:37:04.131811 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3168efb-4d8e-4ba8-b888-7aae6dd1dabf admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:37:04.179353 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 820/1624] 10.222.0.33 () {60 vars in 1143 bytes} [Mon Jul 25 05:37:04 2022] POST /identity/v3/users => generated 288 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 05:37:04.183657 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3168efb-4d8e-4ba8-b888-7aae6dd1dabf admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:37:04.189731 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fc8a29a-29d8-467a-b6a1-53b420a3743c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:37:04.191134 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fc8a29a-29d8-467a-b6a1-53b420a3743c service nova] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:37:04.292628 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 805/1625] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jul 25 05:37:04 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 05:37:04.292628 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc8a29a-29d8-467a-b6a1-53b420a3743c service nova] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:37:04.292628 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc8a29a-29d8-467a-b6a1-53b420a3743c service nova] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:37:04.294014 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3168efb-4d8e-4ba8-b888-7aae6dd1dabf admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:37:04.294554 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c21c2ca-fe37-4010-9bbf-8e3046ec7f27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:37:04.294554 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c21c2ca-fe37-4010-9bbf-8e3046ec7f27 admin admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:37:04.294554 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c21c2ca-fe37-4010-9bbf-8e3046ec7f27 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:37:04.294554 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c21c2ca-fe37-4010-9bbf-8e3046ec7f27 admin admin] PATH_INFO: `/v3/roles` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:37:04.294951 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c21c2ca-fe37-4010-9bbf-8e3046ec7f27 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:37:04.294951 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c21c2ca-fe37-4010-9bbf-8e3046ec7f27 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:37:04.294951 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 806/1626] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jul 25 05:37:04 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 05:37:04.294951 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c21c2ca-fe37-4010-9bbf-8e3046ec7f27 admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:37:04.295253 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-762ab290-4f6e-4657-a5af-e950c2a631a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:37:04.295253 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762ab290-4f6e-4657-a5af-e950c2a631a6 admin admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:37:04.295253 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762ab290-4f6e-4657-a5af-e950c2a631a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:37:04.295253 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762ab290-4f6e-4657-a5af-e950c2a631a6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:37:04.295550 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-762ab290-4f6e-4657-a5af-e950c2a631a6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:37:04.295550 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-762ab290-4f6e-4657-a5af-e950c2a631a6 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:37:04.295691 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c21c2ca-fe37-4010-9bbf-8e3046ec7f27 admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:37:04.301396 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb133945-7b4c-4b1d-9486-85f020163436 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:37:04.302303 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb133945-7b4c-4b1d-9486-85f020163436 admin admin] REQUEST_METHOD: `PUT` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:37:04.302590 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb133945-7b4c-4b1d-9486-85f020163436 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:37:04.302878 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb133945-7b4c-4b1d-9486-85f020163436 admin admin] PATH_INFO: `/v3/projects/f8deb340b1b14caab9fa42b11064d309/users/f49a05a465134ab58f90c2877f830207/roles/39e5c666285a42fe8fc2c3afca6260d1` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:37:04.306709 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:37:04.320254 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb133945-7b4c-4b1d-9486-85f020163436 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f8deb340b1b14caab9fa42b11064d309, user_id=f49a05a465134ab58f90c2877f830207, role_id=39e5c666285a42fe8fc2c3afca6260d1)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:37:04.323056 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb133945-7b4c-4b1d-9486-85f020163436 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:37:04.336072 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 821/1627] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jul 25 05:37:04 2022] GET /identity/v3/roles => generated 1420 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 05:37:04.342507 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-762ab290-4f6e-4657-a5af-e950c2a631a6 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:37:04.349980 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 807/1628] 10.222.0.33 () {60 vars in 1482 bytes} [Mon Jul 25 05:37:04 2022] PUT /identity/v3/projects/f8deb340b1b14caab9fa42b11064d309/users/f49a05a465134ab58f90c2877f830207/roles/39e5c666285a42fe8fc2c3afca6260d1 => generated 0 bytes in 74 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 05:37:04.351106 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb133945-7b4c-4b1d-9486-85f020163436 admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:37:04.355040 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-762ab290-4f6e-4657-a5af-e950c2a631a6 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:37:04.450528 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb133945-7b4c-4b1d-9486-85f020163436 admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:37:04.461674 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e6fbd0-28a1-4679-b97f-bf024fb1bf17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:37:04.463650 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e6fbd0-28a1-4679-b97f-bf024fb1bf17 admin admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:37:04.464796 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e6fbd0-28a1-4679-b97f-bf024fb1bf17 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:37:04.465292 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e6fbd0-28a1-4679-b97f-bf024fb1bf17 admin admin] PATH_INFO: `/v3/roles` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:37:04.466251 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06e6fbd0-28a1-4679-b97f-bf024fb1bf17 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:37:04.468159 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06e6fbd0-28a1-4679-b97f-bf024fb1bf17 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:37:04.492476 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 808/1629] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jul 25 05:37:04 2022] GET /identity/v3/roles => generated 1420 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 05:37:04.494674 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e6fbd0-28a1-4679-b97f-bf024fb1bf17 admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:37:04.516577 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e6fbd0-28a1-4679-b97f-bf024fb1bf17 admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:37:04.529894 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb646df9-7a9c-4c9b-b4f1-f7082406c729 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:37:04.530355 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f16a33c-6cd0-4bfb-883b-4b89f9b30286 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:37:04.530812 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb646df9-7a9c-4c9b-b4f1-f7082406c729 admin admin] REQUEST_METHOD: `PUT` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:37:04.532154 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f16a33c-6cd0-4bfb-883b-4b89f9b30286 admin admin] REQUEST_METHOD: `PUT` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:37:04.532449 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f16a33c-6cd0-4bfb-883b-4b89f9b30286 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:37:04.532732 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f16a33c-6cd0-4bfb-883b-4b89f9b30286 admin admin] PATH_INFO: `/v3/projects/a804b44c02014f74ad2a68f69b9f6489/users/19438e31659248d38999f9500cbb4465/roles/39e5c666285a42fe8fc2c3afca6260d1` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:37:04.534216 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb646df9-7a9c-4c9b-b4f1-f7082406c729 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:37:04.534546 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb646df9-7a9c-4c9b-b4f1-f7082406c729 admin admin] PATH_INFO: `/v3/projects/f8deb340b1b14caab9fa42b11064d309/users/f49a05a465134ab58f90c2877f830207/roles/39e5c666285a42fe8fc2c3afca6260d1` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:37:04.538445 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb646df9-7a9c-4c9b-b4f1-f7082406c729 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f8deb340b1b14caab9fa42b11064d309, user_id=f49a05a465134ab58f90c2877f830207, role_id=39e5c666285a42fe8fc2c3afca6260d1)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:37:04.548125 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb646df9-7a9c-4c9b-b4f1-f7082406c729 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:37:04.589392 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 809/1630] 10.222.0.33 () {60 vars in 1482 bytes} [Mon Jul 25 05:37:04 2022] PUT /identity/v3/projects/f8deb340b1b14caab9fa42b11064d309/users/f49a05a465134ab58f90c2877f830207/roles/39e5c666285a42fe8fc2c3afca6260d1 => generated 0 bytes in 81 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 05:37:04.591819 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376a6b10-55c3-4311-ad9d-e629133419f8 None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:37:04.592258 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376a6b10-55c3-4311-ad9d-e629133419f8 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:37:04.592566 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376a6b10-55c3-4311-ad9d-e629133419f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:37:04.599330 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f16a33c-6cd0-4bfb-883b-4b89f9b30286 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a804b44c02014f74ad2a68f69b9f6489, user_id=19438e31659248d38999f9500cbb4465, role_id=39e5c666285a42fe8fc2c3afca6260d1)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:37:04.607768 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f16a33c-6cd0-4bfb-883b-4b89f9b30286 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:37:04.641957 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 822/1631] 10.222.0.33 () {60 vars in 1482 bytes} [Mon Jul 25 05:37:04 2022] PUT /identity/v3/projects/a804b44c02014f74ad2a68f69b9f6489/users/19438e31659248d38999f9500cbb4465/roles/39e5c666285a42fe8fc2c3afca6260d1 => generated 0 bytes in 301 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 05:37:04.646521 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f16a33c-6cd0-4bfb-883b-4b89f9b30286 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:37:04.666795 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-376a6b10-55c3-4311-ad9d-e629133419f8 None None] MFA Rules not processed for user `f49a05a465134ab58f90c2877f830207`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:37:04.675432 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-376a6b10-55c3-4311-ad9d-e629133419f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:37:04.772835 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f16a33c-6cd0-4bfb-883b-4b89f9b30286 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:37:04.779135 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09bcbbeb-a6b3-405d-b493-e1917a7ffbd8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:37:04.780086 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09bcbbeb-a6b3-405d-b493-e1917a7ffbd8 admin admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:37:04.780534 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09bcbbeb-a6b3-405d-b493-e1917a7ffbd8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:37:04.781325 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09bcbbeb-a6b3-405d-b493-e1917a7ffbd8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:37:04.782250 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09bcbbeb-a6b3-405d-b493-e1917a7ffbd8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:37:04.784065 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09bcbbeb-a6b3-405d-b493-e1917a7ffbd8 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:37:04.803257 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 823/1632] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jul 25 05:37:04 2022] GET /identity/v3/roles => generated 1420 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 05:37:04.811826 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365855b1-b93c-4572-afa2-682d06d5cf14 None None] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:37:04.815315 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365855b1-b93c-4572-afa2-682d06d5cf14 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:37:04.816810 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365855b1-b93c-4572-afa2-682d06d5cf14 None None] PATH_INFO: `/` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:37:04.824143 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 824/1633] 10.222.0.33 () {54 vars in 912 bytes} [Mon Jul 25 05:37:04 2022] GET /identity => generated 270 bytes in 15 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:37:04.825405 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-365855b1-b93c-4572-afa2-682d06d5cf14 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:37:04.843258 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:37:04.853207 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 810/1634] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 25 05:37:04 2022] POST /identity/v3/auth/tokens => generated 3297 bytes in 263 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:37:04.855427 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72b2f15-587b-401a-977a-99cc0d3aab23 None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:37:04.855964 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72b2f15-587b-401a-977a-99cc0d3aab23 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:37:04.856469 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72b2f15-587b-401a-977a-99cc0d3aab23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:37:04.867532 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-365855b1-b93c-4572-afa2-682d06d5cf14 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:37:04.880738 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31fc5195-e252-4934-9ad0-d33f53200e5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:37:04.882461 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31fc5195-e252-4934-9ad0-d33f53200e5e admin admin] REQUEST_METHOD: `PUT` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:37:04.885068 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31fc5195-e252-4934-9ad0-d33f53200e5e admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:37:04.885614 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31fc5195-e252-4934-9ad0-d33f53200e5e admin admin] PATH_INFO: `/v3/projects/a804b44c02014f74ad2a68f69b9f6489/users/19438e31659248d38999f9500cbb4465/roles/39e5c666285a42fe8fc2c3afca6260d1` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:37:04.891531 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31fc5195-e252-4934-9ad0-d33f53200e5e admin admin] RBAC: Authorizing `identity:create_grant(project_id=a804b44c02014f74ad2a68f69b9f6489, user_id=19438e31659248d38999f9500cbb4465, role_id=39e5c666285a42fe8fc2c3afca6260d1)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:37:04.897972 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31fc5195-e252-4934-9ad0-d33f53200e5e admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:37:04.910175 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-f72b2f15-587b-401a-977a-99cc0d3aab23 None None] MFA Rules not processed for user `1cd482a61e8a420db45bb532e5dae6e2`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:37:04.912218 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-f72b2f15-587b-401a-977a-99cc0d3aab23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:37:04.934666 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 825/1635] 10.222.0.33 () {60 vars in 1482 bytes} [Mon Jul 25 05:37:04 2022] PUT /identity/v3/projects/a804b44c02014f74ad2a68f69b9f6489/users/19438e31659248d38999f9500cbb4465/roles/39e5c666285a42fe8fc2c3afca6260d1 => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 05:37:04.935870 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31fc5195-e252-4934-9ad0-d33f53200e5e admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:37:05.035054 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:37:05.060529 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31fc5195-e252-4934-9ad0-d33f53200e5e admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:37:05.069094 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef7321c3-ece5-48ab-b51a-70847a045353 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db57b478c25643dbaf936f90f6e17d0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92afba266cce4d1bad4134c351e74be3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:37:05.069789 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7321c3-ece5-48ab-b51a-70847a045353 service cinder] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:37:05.070081 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7321c3-ece5-48ab-b51a-70847a045353 service cinder] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:37:05.070366 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7321c3-ece5-48ab-b51a-70847a045353 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:37:05.076315 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 811/1636] 10.222.0.33 () {58 vars in 1009 bytes} [Mon Jul 25 05:37:04 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 222 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:37:05.079475 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266418ca-ebbf-488b-861f-f77452309b90 None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:37:05.079984 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266418ca-ebbf-488b-861f-f77452309b90 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:37:05.080437 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266418ca-ebbf-488b-861f-f77452309b90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:37:05.135090 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-266418ca-ebbf-488b-861f-f77452309b90 None None] MFA Rules not processed for user `19438e31659248d38999f9500cbb4465`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:37:05.137742 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-266418ca-ebbf-488b-861f-f77452309b90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:37:05.193205 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef7321c3-ece5-48ab-b51a-70847a045353 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:37:05.194419 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef7321c3-ece5-48ab-b51a-70847a045353 service cinder] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:37:05.210948 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 826/1637] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jul 25 05:37:04 2022] GET /identity/v3/auth/tokens => generated 3297 bytes in 276 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 05:37:05.212432 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d10222-2694-444a-946c-ba019f82460b None None] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:37:05.212729 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d10222-2694-444a-946c-ba019f82460b None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:37:05.213075 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d10222-2694-444a-946c-ba019f82460b None None] PATH_INFO: `/` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:37:05.214944 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 827/1638] 10.222.0.33 () {54 vars in 912 bytes} [Mon Jul 25 05:37:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:37:05.215666 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d10222-2694-444a-946c-ba019f82460b None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:37:05.232677 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d10222-2694-444a-946c-ba019f82460b None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:37:05.237071 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd96414-cff4-4000-92ed-7799c51bb3cb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db57b478c25643dbaf936f90f6e17d0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92afba266cce4d1bad4134c351e74be3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:37:05.237840 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd96414-cff4-4000-92ed-7799c51bb3cb service cinder] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:37:05.238183 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd96414-cff4-4000-92ed-7799c51bb3cb service cinder] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:37:05.238539 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd96414-cff4-4000-92ed-7799c51bb3cb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:37:05.261248 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dd96414-cff4-4000-92ed-7799c51bb3cb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:37:05.262749 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dd96414-cff4-4000-92ed-7799c51bb3cb service cinder] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:37:05.269340 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 812/1639] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 25 05:37:05 2022] POST /identity/v3/auth/tokens => generated 3337 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:37:05.271558 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69955659-f888-42b9-a869-c8aa58f7fb65 None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:37:05.272000 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69955659-f888-42b9-a869-c8aa58f7fb65 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:37:05.272834 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69955659-f888-42b9-a869-c8aa58f7fb65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:37:05.297188 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-69955659-f888-42b9-a869-c8aa58f7fb65 None None] MFA Rules not processed for user `1cd482a61e8a420db45bb532e5dae6e2`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:37:05.298801 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-69955659-f888-42b9-a869-c8aa58f7fb65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:37:05.310298 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 828/1640] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jul 25 05:37:05 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 05:37:05.311500 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd96414-cff4-4000-92ed-7799c51bb3cb service cinder] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:37:05.316560 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 813/1641] 10.222.0.33 () {58 vars in 1009 bytes} [Mon Jul 25 05:37:05 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:37:05.331647 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd96414-cff4-4000-92ed-7799c51bb3cb service cinder] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:37:05.340902 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c0eb927-e276-4fe4-bc40-e3e43094b372 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db57b478c25643dbaf936f90f6e17d0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92afba266cce4d1bad4134c351e74be3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:37:05.341955 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0eb927-e276-4fe4-bc40-e3e43094b372 service cinder] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:37:05.344465 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0eb927-e276-4fe4-bc40-e3e43094b372 service cinder] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:37:05.344757 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0eb927-e276-4fe4-bc40-e3e43094b372 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:37:05.361244 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c0eb927-e276-4fe4-bc40-e3e43094b372 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:37:05.362860 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c0eb927-e276-4fe4-bc40-e3e43094b372 service cinder] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:37:05.374338 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 829/1642] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jul 25 05:37:05 2022] GET /identity/v3/auth/tokens => generated 3337 bytes in 63 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 05:37:05.578871 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69955659-f888-42b9-a869-c8aa58f7fb65 None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:37:05.598106 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69955659-f888-42b9-a869-c8aa58f7fb65 None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:37:05.602031 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485e37ff-799f-44c9-ad99-f2a998294fae service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db57b478c25643dbaf936f90f6e17d0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92afba266cce4d1bad4134c351e74be3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:37:05.602690 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485e37ff-799f-44c9-ad99-f2a998294fae service cinder] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:37:05.602972 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485e37ff-799f-44c9-ad99-f2a998294fae service cinder] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:37:05.603245 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485e37ff-799f-44c9-ad99-f2a998294fae service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:37:05.621686 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-485e37ff-799f-44c9-ad99-f2a998294fae service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:37:05.626503 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-485e37ff-799f-44c9-ad99-f2a998294fae service cinder] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:37:05.664439 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 814/1643] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jul 25 05:37:05 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 05:37:06.547863 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c0eb927-e276-4fe4-bc40-e3e43094b372 service cinder] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:37:06.562411 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c0eb927-e276-4fe4-bc40-e3e43094b372 service cinder] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:37:06.576935 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd35e2f0-a925-4338-bf13-01b2fdf2c295 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db57b478c25643dbaf936f90f6e17d0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92afba266cce4d1bad4134c351e74be3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:37:06.577940 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd35e2f0-a925-4338-bf13-01b2fdf2c295 service cinder] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:37:06.579977 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd35e2f0-a925-4338-bf13-01b2fdf2c295 service cinder] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:37:06.580472 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd35e2f0-a925-4338-bf13-01b2fdf2c295 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:37:06.756965 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd35e2f0-a925-4338-bf13-01b2fdf2c295 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:37:06.760036 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd35e2f0-a925-4338-bf13-01b2fdf2c295 service cinder] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:37:06.798011 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 830/1644] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jul 25 05:37:06 2022] GET /identity/v3/auth/tokens => generated 3230 bytes in 251 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 05:37:07.216401 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485e37ff-799f-44c9-ad99-f2a998294fae service cinder] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:37:07.346244 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485e37ff-799f-44c9-ad99-f2a998294fae service cinder] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:37:07.353065 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79f960a-c83e-4703-8d47-ea6f7fd1c7d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:37:07.354029 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79f960a-c83e-4703-8d47-ea6f7fd1c7d7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:37:07.354320 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79f960a-c83e-4703-8d47-ea6f7fd1c7d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:37:07.354709 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79f960a-c83e-4703-8d47-ea6f7fd1c7d7 admin admin] PATH_INFO: `/v3/users/f2008c3f679441428a89babc69b76a7d` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:37:07.357523 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d79f960a-c83e-4703-8d47-ea6f7fd1c7d7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f2008c3f679441428a89babc69b76a7d)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:37:07.359524 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: WARNING py.warnings [None req-d79f960a-c83e-4703-8d47-ea6f7fd1c7d7 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 25 05:37:07.359524 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: warnings.warn(msg) Jul 25 05:37:07.359524 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: Jul 25 05:37:07.361661 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d79f960a-c83e-4703-8d47-ea6f7fd1c7d7 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:37:07.554256 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-d79f960a-c83e-4703-8d47-ea6f7fd1c7d7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f2008c3f679441428a89babc69b76a7d'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:37:07.569015 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02795e85-129e-418b-b063-639fde2857b1 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:37:07.570265 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02795e85-129e-418b-b063-639fde2857b1 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:37:07.570714 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02795e85-129e-418b-b063-639fde2857b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:37:07.579599 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:37:07.597643 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-d79f960a-c83e-4703-8d47-ea6f7fd1c7d7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f2008c3f679441428a89babc69b76a7d'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:37:07.600212 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-d79f960a-c83e-4703-8d47-ea6f7fd1c7d7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f2008c3f679441428a89babc69b76a7d'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:37:07.615720 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-d79f960a-c83e-4703-8d47-ea6f7fd1c7d7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f2008c3f679441428a89babc69b76a7d'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:37:07.616553 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-d79f960a-c83e-4703-8d47-ea6f7fd1c7d7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f2008c3f679441428a89babc69b76a7d'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:37:07.625760 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:37:07.631409 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-02795e85-129e-418b-b063-639fde2857b1 None None] MFA Rules not processed for user `aec7125a032f42d19b106b79c9968d46`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:37:07.635354 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-02795e85-129e-418b-b063-639fde2857b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:37:07.658430 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 815/1645] 10.222.0.33 () {58 vars in 1223 bytes} [Mon Jul 25 05:37:07 2022] DELETE /identity/v3/users/f2008c3f679441428a89babc69b76a7d => generated 0 bytes in 442 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 05:37:07.664999 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ab3aca-d80a-45e0-83b7-78307fd7a20a None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:37:07.665491 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ab3aca-d80a-45e0-83b7-78307fd7a20a None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:37:07.665927 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ab3aca-d80a-45e0-83b7-78307fd7a20a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:37:07.691371 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-15ab3aca-d80a-45e0-83b7-78307fd7a20a None None] MFA Rules not processed for user `1eae540242d94c10b0ad33960a855f08`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:37:07.694070 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-15ab3aca-d80a-45e0-83b7-78307fd7a20a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:37:07.795973 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 831/1646] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 25 05:37:07 2022] POST /identity/v3/auth/tokens => generated 3272 bytes in 226 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 05:37:07.797194 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02795e85-129e-418b-b063-639fde2857b1 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:37:07.799668 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-02795e85-129e-418b-b063-639fde2857b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:37:07.821418 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02795e85-129e-418b-b063-639fde2857b1 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:37:07.852194 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 816/1647] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 25 05:37:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:37:07.905911 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-700d8358-f7fe-40bf-8a28-4e6e5eeaf48e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:37:07.906681 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700d8358-f7fe-40bf-8a28-4e6e5eeaf48e admin admin] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:37:07.907047 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700d8358-f7fe-40bf-8a28-4e6e5eeaf48e admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:37:07.907542 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700d8358-f7fe-40bf-8a28-4e6e5eeaf48e admin admin] PATH_INFO: `/v3/projects` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:37:07.908332 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-700d8358-f7fe-40bf-8a28-4e6e5eeaf48e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:37:07.910430 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-700d8358-f7fe-40bf-8a28-4e6e5eeaf48e admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:37:07.940479 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ab3aca-d80a-45e0-83b7-78307fd7a20a None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:37:07.942516 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-15ab3aca-d80a-45e0-83b7-78307fd7a20a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:37:07.956853 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 832/1648] 10.222.0.33 () {60 vars in 1152 bytes} [Mon Jul 25 05:37:07 2022] POST /identity/v3/projects => generated 405 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 05:37:07.959007 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-700d8358-f7fe-40bf-8a28-4e6e5eeaf48e admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:37:08.079576 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ab3aca-d80a-45e0-83b7-78307fd7a20a None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:37:08.126033 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-700d8358-f7fe-40bf-8a28-4e6e5eeaf48e admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:37:08.144216 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01fe8deb-957b-4b58-8306-3bb4588867f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:37:08.145043 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fe8deb-957b-4b58-8306-3bb4588867f4 admin admin] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:37:08.145436 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fe8deb-957b-4b58-8306-3bb4588867f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:37:08.145822 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fe8deb-957b-4b58-8306-3bb4588867f4 admin admin] PATH_INFO: `/v3/users` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:37:08.146647 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01fe8deb-957b-4b58-8306-3bb4588867f4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:37:08.150469 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01fe8deb-957b-4b58-8306-3bb4588867f4 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:37:08.198307 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 833/1649] 10.222.0.33 () {60 vars in 1143 bytes} [Mon Jul 25 05:37:07 2022] POST /identity/v3/users => generated 309 bytes in 237 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 05:37:08.199982 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01fe8deb-957b-4b58-8306-3bb4588867f4 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:37:08.222435 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a7318e-c30a-4280-a949-e0d64f7367b2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b2931200f99e43d79a50a9f6e02738e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92afba266cce4d1bad4134c351e74be3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:37:08.223002 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a7318e-c30a-4280-a949-e0d64f7367b2 service neutron] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:37:08.223218 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a7318e-c30a-4280-a949-e0d64f7367b2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:37:08.223499 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a7318e-c30a-4280-a949-e0d64f7367b2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:37:08.227174 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01fe8deb-957b-4b58-8306-3bb4588867f4 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:37:08.239051 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d0bc24-610b-4cd6-9bba-16b0c80131b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:37:08.244839 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d0bc24-610b-4cd6-9bba-16b0c80131b8 admin admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:37:08.246324 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d0bc24-610b-4cd6-9bba-16b0c80131b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:37:08.247159 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4a7318e-c30a-4280-a949-e0d64f7367b2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:37:08.247464 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d0bc24-610b-4cd6-9bba-16b0c80131b8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:37:08.248554 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4d0bc24-610b-4cd6-9bba-16b0c80131b8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:37:08.248682 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4a7318e-c30a-4280-a949-e0d64f7367b2 service neutron] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:37:08.254623 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4d0bc24-610b-4cd6-9bba-16b0c80131b8 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:37:08.316149 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 817/1650] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jul 25 05:37:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 25 05:37:08.316708 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 834/1651] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jul 25 05:37:08 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 05:37:08.317564 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a7318e-c30a-4280-a949-e0d64f7367b2 service neutron] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:37:08.335177 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a7318e-c30a-4280-a949-e0d64f7367b2 service neutron] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:37:08.349224 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e1fcdf3-922c-4bb9-b8ef-401bf27accee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:37:08.350616 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1fcdf3-922c-4bb9-b8ef-401bf27accee admin admin] REQUEST_METHOD: `PUT` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:37:08.350616 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1fcdf3-922c-4bb9-b8ef-401bf27accee admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:37:08.350616 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1fcdf3-922c-4bb9-b8ef-401bf27accee admin admin] PATH_INFO: `/v3/projects/20403937bb684d8e9d29b6992bf4e9f6/users/dd9bcd70fa9a4c27a2e1eb72218ca242/roles/39e5c666285a42fe8fc2c3afca6260d1` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:37:08.388543 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e1fcdf3-922c-4bb9-b8ef-401bf27accee admin admin] RBAC: Authorizing `identity:create_grant(project_id=20403937bb684d8e9d29b6992bf4e9f6, user_id=dd9bcd70fa9a4c27a2e1eb72218ca242, role_id=39e5c666285a42fe8fc2c3afca6260d1)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:37:08.391888 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e1fcdf3-922c-4bb9-b8ef-401bf27accee admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:37:08.436353 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 818/1652] 10.222.0.33 () {60 vars in 1482 bytes} [Mon Jul 25 05:37:08 2022] PUT /identity/v3/projects/20403937bb684d8e9d29b6992bf4e9f6/users/dd9bcd70fa9a4c27a2e1eb72218ca242/roles/39e5c666285a42fe8fc2c3afca6260d1 => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 05:37:08.441016 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d0bc24-610b-4cd6-9bba-16b0c80131b8 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:37:08.463591 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d0bc24-610b-4cd6-9bba-16b0c80131b8 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:37:08.469480 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:37:08.660874 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e4a1dc3-d266-43d7-96a2-0538a6ecbb46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:37:08.662696 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4a1dc3-d266-43d7-96a2-0538a6ecbb46 admin admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:37:08.663002 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4a1dc3-d266-43d7-96a2-0538a6ecbb46 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:37:08.663394 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4a1dc3-d266-43d7-96a2-0538a6ecbb46 admin admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:37:08.664177 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e4a1dc3-d266-43d7-96a2-0538a6ecbb46 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:37:08.671662 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e4a1dc3-d266-43d7-96a2-0538a6ecbb46 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:37:08.705588 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 835/1653] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jul 25 05:37:08 2022] GET /identity/v3/roles => generated 1420 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 05:37:08.709525 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e1fcdf3-922c-4bb9-b8ef-401bf27accee admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:37:08.716891 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:DELETE=3,keystone:SELECT=95,keystone:INSERT=2 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:37:08.726249 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e1fcdf3-922c-4bb9-b8ef-401bf27accee admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:37:08.744319 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c12338-19c8-41ba-b110-2126a7a26003 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:37:08.746124 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c12338-19c8-41ba-b110-2126a7a26003 admin admin] REQUEST_METHOD: `PUT` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:37:08.749668 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c12338-19c8-41ba-b110-2126a7a26003 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:37:08.750472 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c12338-19c8-41ba-b110-2126a7a26003 admin admin] PATH_INFO: `/v3/projects/20403937bb684d8e9d29b6992bf4e9f6/users/dd9bcd70fa9a4c27a2e1eb72218ca242/roles/39e5c666285a42fe8fc2c3afca6260d1` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:37:08.761856 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04c12338-19c8-41ba-b110-2126a7a26003 admin admin] RBAC: Authorizing `identity:create_grant(project_id=20403937bb684d8e9d29b6992bf4e9f6, user_id=dd9bcd70fa9a4c27a2e1eb72218ca242, role_id=39e5c666285a42fe8fc2c3afca6260d1)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:37:08.772029 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04c12338-19c8-41ba-b110-2126a7a26003 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:37:08.796491 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 819/1654] 10.222.0.33 () {60 vars in 1482 bytes} [Mon Jul 25 05:37:08 2022] PUT /identity/v3/projects/20403937bb684d8e9d29b6992bf4e9f6/users/dd9bcd70fa9a4c27a2e1eb72218ca242/roles/39e5c666285a42fe8fc2c3afca6260d1 => generated 0 bytes in 88 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 05:37:10.344203 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45fc2249-51d2-451b-8045-6d0c0782f7d2 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:37:10.344741 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45fc2249-51d2-451b-8045-6d0c0782f7d2 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:37:10.345156 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45fc2249-51d2-451b-8045-6d0c0782f7d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:37:10.410957 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-45fc2249-51d2-451b-8045-6d0c0782f7d2 None None] MFA Rules not processed for user `37c1bb8500f74013b073febfc0d96217`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:37:10.412328 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-45fc2249-51d2-451b-8045-6d0c0782f7d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:37:10.626613 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 836/1655] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 25 05:37:10 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 277 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:37:10.629400 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0816987-be27-4508-9ac1-700f743e35ee None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:37:10.629955 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0816987-be27-4508-9ac1-700f743e35ee None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:37:10.630089 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0816987-be27-4508-9ac1-700f743e35ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:37:10.675276 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-a0816987-be27-4508-9ac1-700f743e35ee None None] MFA Rules not processed for user `37c1bb8500f74013b073febfc0d96217`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:37:10.676550 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-a0816987-be27-4508-9ac1-700f743e35ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:37:10.691100 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 820/1656] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 25 05:37:10 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 59 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:37:10.699144 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45fc2249-51d2-451b-8045-6d0c0782f7d2 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:37:10.700591 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-45fc2249-51d2-451b-8045-6d0c0782f7d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:37:10.854134 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45fc2249-51d2-451b-8045-6d0c0782f7d2 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:37:10.862838 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f4e722-5eed-4bb2-8382-c3f226c5c3f9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cd482a61e8a420db45bb532e5dae6e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92afba266cce4d1bad4134c351e74be3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:37:10.863686 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f4e722-5eed-4bb2-8382-c3f226c5c3f9 service nova] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:37:10.864084 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f4e722-5eed-4bb2-8382-c3f226c5c3f9 service nova] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:37:10.864469 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f4e722-5eed-4bb2-8382-c3f226c5c3f9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:37:10.879664 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1f4e722-5eed-4bb2-8382-c3f226c5c3f9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:37:10.882860 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1f4e722-5eed-4bb2-8382-c3f226c5c3f9 service nova] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:37:10.915756 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0816987-be27-4508-9ac1-700f743e35ee None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:37:10.919819 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-a0816987-be27-4508-9ac1-700f743e35ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:37:10.959465 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 837/1657] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jul 25 05:37:10 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 262 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 05:37:11.115142 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0816987-be27-4508-9ac1-700f743e35ee None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:37:11.122528 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db2508fc-455b-42f1-b25c-fa204a2d39ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:37:11.123720 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2508fc-455b-42f1-b25c-fa204a2d39ee admin admin] REQUEST_METHOD: `DELETE` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:37:11.124230 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2508fc-455b-42f1-b25c-fa204a2d39ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:37:11.124704 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2508fc-455b-42f1-b25c-fa204a2d39ee admin admin] PATH_INFO: `/v3/projects/2854c008a6674dc1b63c6550d0f2108f` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:37:11.126308 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db2508fc-455b-42f1-b25c-fa204a2d39ee admin admin] RBAC: Authorizing `identity:delete_project(project_id=2854c008a6674dc1b63c6550d0f2108f)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:37:11.128502 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db2508fc-455b-42f1-b25c-fa204a2d39ee admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:37:11.215253 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-db2508fc-455b-42f1-b25c-fa204a2d39ee admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2854c008a6674dc1b63c6550d0f2108f', 'request_id': 'req-db2508fc-455b-42f1-b25c-fa204a2d39ee'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:37:11.227168 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-db2508fc-455b-42f1-b25c-fa204a2d39ee admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2854c008a6674dc1b63c6550d0f2108f', 'request_id': 'req-db2508fc-455b-42f1-b25c-fa204a2d39ee'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:37:11.241915 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-db2508fc-455b-42f1-b25c-fa204a2d39ee admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2854c008a6674dc1b63c6550d0f2108f', 'request_id': 'req-db2508fc-455b-42f1-b25c-fa204a2d39ee'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:37:11.270861 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-db2508fc-455b-42f1-b25c-fa204a2d39ee admin admin] The token cache is being invalidate because project 2854c008a6674dc1b63c6550d0f2108f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80248) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 05:37:11.270861 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-db2508fc-455b-42f1-b25c-fa204a2d39ee admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:37:11.272682 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-db2508fc-455b-42f1-b25c-fa204a2d39ee admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:37:11.275406 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 821/1658] 10.222.0.33 () {58 vars in 1232 bytes} [Mon Jul 25 05:37:10 2022] DELETE /identity/v3/projects/2854c008a6674dc1b63c6550d0f2108f => generated 0 bytes in 360 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 05:37:11.279987 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f4e722-5eed-4bb2-8382-c3f226c5c3f9 service nova] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:37:11.297985 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f4e722-5eed-4bb2-8382-c3f226c5c3f9 service nova] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:37:11.340314 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:37:11.474315 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39cf8717-0dbd-4a23-bd46-3b83389001e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:37:11.475148 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39cf8717-0dbd-4a23-bd46-3b83389001e9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:37:11.475443 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39cf8717-0dbd-4a23-bd46-3b83389001e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:37:11.475728 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39cf8717-0dbd-4a23-bd46-3b83389001e9 admin admin] PATH_INFO: `/v3/users/19438e31659248d38999f9500cbb4465` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:37:11.477003 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39cf8717-0dbd-4a23-bd46-3b83389001e9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=19438e31659248d38999f9500cbb4465)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:37:11.481099 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39cf8717-0dbd-4a23-bd46-3b83389001e9 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:37:11.615969 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-39cf8717-0dbd-4a23-bd46-3b83389001e9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '19438e31659248d38999f9500cbb4465'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:37:11.617172 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-39cf8717-0dbd-4a23-bd46-3b83389001e9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '19438e31659248d38999f9500cbb4465'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:37:11.638892 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-39cf8717-0dbd-4a23-bd46-3b83389001e9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '19438e31659248d38999f9500cbb4465'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:37:11.677928 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-39cf8717-0dbd-4a23-bd46-3b83389001e9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '19438e31659248d38999f9500cbb4465'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:37:11.677928 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-39cf8717-0dbd-4a23-bd46-3b83389001e9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '19438e31659248d38999f9500cbb4465'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:37:11.679420 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ef1f13-ca03-4dab-9f7e-1caa85173304 None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:37:11.679984 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ef1f13-ca03-4dab-9f7e-1caa85173304 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:37:11.680227 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ef1f13-ca03-4dab-9f7e-1caa85173304 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:37:11.710036 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 838/1659] 10.222.0.33 () {58 vars in 1223 bytes} [Mon Jul 25 05:37:11 2022] DELETE /identity/v3/users/19438e31659248d38999f9500cbb4465 => generated 0 bytes in 431 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 05:37:11.746024 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-74ef1f13-ca03-4dab-9f7e-1caa85173304 None None] MFA Rules not processed for user `dd9bcd70fa9a4c27a2e1eb72218ca242`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:37:11.747205 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-74ef1f13-ca03-4dab-9f7e-1caa85173304 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:37:11.886623 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bbfa691-73dc-4f68-9508-a0e6e637eb21 None None] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:37:11.888139 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bbfa691-73dc-4f68-9508-a0e6e637eb21 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:37:11.889144 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bbfa691-73dc-4f68-9508-a0e6e637eb21 None None] PATH_INFO: `/` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:37:11.895381 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 839/1660] 10.222.0.33 () {54 vars in 912 bytes} [Mon Jul 25 05:37:11 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:37:11.910783 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaaf3c89-3af3-401b-a309-aeeb601af97b None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:37:11.911340 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaaf3c89-3af3-401b-a309-aeeb601af97b None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:37:11.911799 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaaf3c89-3af3-401b-a309-aeeb601af97b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:37:11.923519 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:37:11.927708 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 822/1661] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 25 05:37:11 2022] POST /identity/v3/auth/tokens => generated 3339 bytes in 249 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 05:37:12.002145 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-eaaf3c89-3af3-401b-a309-aeeb601af97b None None] MFA Rules not processed for user `b2931200f99e43d79a50a9f6e02738e6`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:37:12.003666 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-eaaf3c89-3af3-401b-a309-aeeb601af97b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:37:12.155609 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 840/1662] 10.222.0.33 () {58 vars in 1009 bytes} [Mon Jul 25 05:37:11 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 255 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:37:12.228531 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ef1f13-ca03-4dab-9f7e-1caa85173304 None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:37:12.230756 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-74ef1f13-ca03-4dab-9f7e-1caa85173304 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:37:12.265265 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ef1f13-ca03-4dab-9f7e-1caa85173304 None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:37:12.271253 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f8a0396-d1f8-4251-bf3f-97349fc7fdc8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b2931200f99e43d79a50a9f6e02738e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92afba266cce4d1bad4134c351e74be3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:37:12.272185 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8a0396-d1f8-4251-bf3f-97349fc7fdc8 service neutron] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:37:12.272682 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8a0396-d1f8-4251-bf3f-97349fc7fdc8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:37:12.273173 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8a0396-d1f8-4251-bf3f-97349fc7fdc8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:37:12.296586 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f8a0396-d1f8-4251-bf3f-97349fc7fdc8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:37:12.299304 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f8a0396-d1f8-4251-bf3f-97349fc7fdc8 service neutron] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:37:12.322582 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 823/1663] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jul 25 05:37:12 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 25 05:37:12.918146 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaaf3c89-3af3-401b-a309-aeeb601af97b None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:37:12.920924 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-eaaf3c89-3af3-401b-a309-aeeb601af97b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:37:13.016928 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:37:13.093899 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaaf3c89-3af3-401b-a309-aeeb601af97b None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:37:13.124768 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf8f1be-19be-4508-9115-bf35fccca908 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:37:13.125705 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf8f1be-19be-4508-9115-bf35fccca908 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:37:13.126008 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf8f1be-19be-4508-9115-bf35fccca908 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:37:13.126286 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf8f1be-19be-4508-9115-bf35fccca908 admin admin] PATH_INFO: `/v3/projects/a804b44c02014f74ad2a68f69b9f6489` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:37:13.127620 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbf8f1be-19be-4508-9115-bf35fccca908 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a804b44c02014f74ad2a68f69b9f6489)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:37:13.129998 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: WARNING py.warnings [None req-cbf8f1be-19be-4508-9115-bf35fccca908 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 25 05:37:13.129998 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: warnings.warn(msg) Jul 25 05:37:13.129998 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: Jul 25 05:37:13.131630 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbf8f1be-19be-4508-9115-bf35fccca908 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:37:13.256417 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-cbf8f1be-19be-4508-9115-bf35fccca908 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a804b44c02014f74ad2a68f69b9f6489', 'request_id': 'req-cbf8f1be-19be-4508-9115-bf35fccca908'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:37:13.266861 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-cbf8f1be-19be-4508-9115-bf35fccca908 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a804b44c02014f74ad2a68f69b9f6489', 'request_id': 'req-cbf8f1be-19be-4508-9115-bf35fccca908'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:37:13.287557 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-cbf8f1be-19be-4508-9115-bf35fccca908 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a804b44c02014f74ad2a68f69b9f6489', 'request_id': 'req-cbf8f1be-19be-4508-9115-bf35fccca908'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:37:13.289711 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b48d595-e6a2-4680-9560-3acb80e14839 None None] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:37:13.290167 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b48d595-e6a2-4680-9560-3acb80e14839 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:37:13.290316 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b48d595-e6a2-4680-9560-3acb80e14839 None None] PATH_INFO: `/` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:37:13.293486 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 824/1664] 10.222.0.33 () {54 vars in 961 bytes} [Mon Jul 25 05:37:13 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:37:13.305351 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222b2167-56fa-41b3-9334-80c32cf45f10 None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:37:13.305649 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222b2167-56fa-41b3-9334-80c32cf45f10 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:37:13.306058 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222b2167-56fa-41b3-9334-80c32cf45f10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:37:13.311043 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-cbf8f1be-19be-4508-9115-bf35fccca908 admin admin] The token cache is being invalidate because project a804b44c02014f74ad2a68f69b9f6489 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80249) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 05:37:13.311439 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-cbf8f1be-19be-4508-9115-bf35fccca908 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:37:13.312315 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-cbf8f1be-19be-4508-9115-bf35fccca908 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:37:13.318474 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 841/1665] 10.222.0.33 () {58 vars in 1232 bytes} [Mon Jul 25 05:37:12 2022] DELETE /identity/v3/projects/a804b44c02014f74ad2a68f69b9f6489 => generated 0 bytes in 399 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 05:37:13.328502 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af3b5b9-70ba-4755-ae2d-a8daae442596 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:37:13.328502 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af3b5b9-70ba-4755-ae2d-a8daae442596 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:37:13.328502 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af3b5b9-70ba-4755-ae2d-a8daae442596 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:37:13.393381 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-222b2167-56fa-41b3-9334-80c32cf45f10 None None] MFA Rules not processed for user `317790016e7047fe96af73925de5d872`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:37:13.398060 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-222b2167-56fa-41b3-9334-80c32cf45f10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:37:13.398963 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-8af3b5b9-70ba-4755-ae2d-a8daae442596 None None] MFA Rules not processed for user `1eae540242d94c10b0ad33960a855f08`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:37:13.400216 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-8af3b5b9-70ba-4755-ae2d-a8daae442596 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:37:13.529084 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 825/1666] 10.222.0.33 () {58 vars in 1058 bytes} [Mon Jul 25 05:37:13 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 225 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 05:37:13.530904 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2fcf7cc-8f76-45f6-9366-0df8a9f96e4b None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:37:13.532840 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2fcf7cc-8f76-45f6-9366-0df8a9f96e4b None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:37:13.533270 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2fcf7cc-8f76-45f6-9366-0df8a9f96e4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:37:13.579313 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 842/1667] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 25 05:37:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 255 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:37:13.580492 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af3b5b9-70ba-4755-ae2d-a8daae442596 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:37:13.593243 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-d2fcf7cc-8f76-45f6-9366-0df8a9f96e4b None None] MFA Rules not processed for user `37c1bb8500f74013b073febfc0d96217`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:37:13.593683 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af3b5b9-70ba-4755-ae2d-a8daae442596 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:37:13.596834 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-d2fcf7cc-8f76-45f6-9366-0df8a9f96e4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:37:13.600205 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0310e79a-18df-489e-b9b6-d1775b7c9d06 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '317790016e7047fe96af73925de5d872', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92afba266cce4d1bad4134c351e74be3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:37:13.601442 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0310e79a-18df-489e-b9b6-d1775b7c9d06 service placement] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:37:13.601735 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0310e79a-18df-489e-b9b6-d1775b7c9d06 service placement] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:37:13.602022 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0310e79a-18df-489e-b9b6-d1775b7c9d06 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:37:13.605148 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-0310e79a-18df-489e-b9b6-d1775b7c9d06 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:37:13.619471 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0310e79a-18df-489e-b9b6-d1775b7c9d06 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:37:13.621756 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0310e79a-18df-489e-b9b6-d1775b7c9d06 service placement] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:37:13.634986 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 843/1668] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jul 25 05:37:13 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 05:37:13.637341 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692f3584-7609-4a48-9910-229bc29ee1b4 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:37:13.637638 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692f3584-7609-4a48-9910-229bc29ee1b4 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:37:13.638114 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692f3584-7609-4a48-9910-229bc29ee1b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:37:13.659303 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-692f3584-7609-4a48-9910-229bc29ee1b4 None None] MFA Rules not processed for user `1eae540242d94c10b0ad33960a855f08`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:37:13.660864 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-692f3584-7609-4a48-9910-229bc29ee1b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:37:13.677355 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 844/1669] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 25 05:37:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:37:13.678648 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-692f3584-7609-4a48-9910-229bc29ee1b4 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:37:13.692852 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-692f3584-7609-4a48-9910-229bc29ee1b4 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:37:13.699607 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d864400c-2880-4324-a5f4-d744461089c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:37:13.700340 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d864400c-2880-4324-a5f4-d744461089c0 admin admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:37:13.700635 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d864400c-2880-4324-a5f4-d744461089c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:37:13.701001 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d864400c-2880-4324-a5f4-d744461089c0 admin admin] PATH_INFO: `/v3/domains` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:37:13.702000 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d864400c-2880-4324-a5f4-d744461089c0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:37:13.702453 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d864400c-2880-4324-a5f4-d744461089c0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:37:13.704528 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d864400c-2880-4324-a5f4-d744461089c0 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:37:13.718080 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 845/1670] 10.222.0.33 () {58 vars in 1152 bytes} [Mon Jul 25 05:37:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:37:13.723567 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d864400c-2880-4324-a5f4-d744461089c0 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:37:13.737127 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 826/1671] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 25 05:37:13 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:37:13.737631 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d864400c-2880-4324-a5f4-d744461089c0 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:37:13.749060 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f823da40-5284-4998-9f41-80b9abb350a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:37:13.750639 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f823da40-5284-4998-9f41-80b9abb350a5 admin admin] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:37:13.751132 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f823da40-5284-4998-9f41-80b9abb350a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:37:13.751557 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f823da40-5284-4998-9f41-80b9abb350a5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:37:13.752334 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f823da40-5284-4998-9f41-80b9abb350a5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:37:13.756208 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f823da40-5284-4998-9f41-80b9abb350a5 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:37:13.812568 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2fcf7cc-8f76-45f6-9366-0df8a9f96e4b None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:37:13.813059 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 846/1672] 10.222.0.33 () {60 vars in 1152 bytes} [Mon Jul 25 05:37:13 2022] POST /identity/v3/projects => generated 377 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 05:37:13.844432 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2fcf7cc-8f76-45f6-9366-0df8a9f96e4b None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:37:14.066846 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-939e5c60-4d15-423c-8588-d10be8b7641f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:37:14.068891 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939e5c60-4d15-423c-8588-d10be8b7641f admin admin] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:37:14.069201 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939e5c60-4d15-423c-8588-d10be8b7641f admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:37:14.069489 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939e5c60-4d15-423c-8588-d10be8b7641f admin admin] PATH_INFO: `/v3/users` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:37:14.070229 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-939e5c60-4d15-423c-8588-d10be8b7641f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:37:14.072223 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-939e5c60-4d15-423c-8588-d10be8b7641f admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:37:14.105682 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f823da40-5284-4998-9f41-80b9abb350a5 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:37:14.109132 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-f823da40-5284-4998-9f41-80b9abb350a5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:37:14.140594 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f823da40-5284-4998-9f41-80b9abb350a5 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:37:14.146538 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e53a135c-9da0-4549-b4c3-5b9a316927ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:37:14.150412 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 827/1673] 10.222.0.33 () {60 vars in 1143 bytes} [Mon Jul 25 05:37:13 2022] POST /identity/v3/users => generated 289 bytes in 336 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 05:37:14.151819 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53a135c-9da0-4549-b4c3-5b9a316927ce admin admin] REQUEST_METHOD: `DELETE` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:37:14.152349 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-939e5c60-4d15-423c-8588-d10be8b7641f admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:37:14.153041 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53a135c-9da0-4549-b4c3-5b9a316927ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:37:14.153565 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53a135c-9da0-4549-b4c3-5b9a316927ce admin admin] PATH_INFO: `/v3/users/316fd0c7342946638423ecfc7a506dfb` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:37:14.155209 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e53a135c-9da0-4549-b4c3-5b9a316927ce admin admin] RBAC: Authorizing `identity:delete_user(user_id=316fd0c7342946638423ecfc7a506dfb)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:37:14.158089 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e53a135c-9da0-4549-b4c3-5b9a316927ce admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:37:14.162966 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:37:14.187162 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-939e5c60-4d15-423c-8588-d10be8b7641f admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:37:14.210994 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fdc1e69-69a1-48f8-a9d7-1d54c6e38191 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:37:14.211489 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fdc1e69-69a1-48f8-a9d7-1d54c6e38191 admin admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:37:14.211783 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fdc1e69-69a1-48f8-a9d7-1d54c6e38191 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:37:14.212063 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fdc1e69-69a1-48f8-a9d7-1d54c6e38191 admin admin] PATH_INFO: `/v3/roles` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:37:14.212826 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fdc1e69-69a1-48f8-a9d7-1d54c6e38191 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:37:14.214520 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fdc1e69-69a1-48f8-a9d7-1d54c6e38191 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:37:14.264977 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 828/1674] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jul 25 05:37:14 2022] GET /identity/v3/roles => generated 1420 bytes in 114 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 05:37:14.269909 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fdc1e69-69a1-48f8-a9d7-1d54c6e38191 admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:37:14.303482 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fdc1e69-69a1-48f8-a9d7-1d54c6e38191 admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:37:14.325567 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b60623-bb9c-4dd5-8a1d-6c0b0b8ed3f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:37:14.326273 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b60623-bb9c-4dd5-8a1d-6c0b0b8ed3f3 admin admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:37:14.326561 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b60623-bb9c-4dd5-8a1d-6c0b0b8ed3f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:37:14.327007 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b60623-bb9c-4dd5-8a1d-6c0b0b8ed3f3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:37:14.327926 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9b60623-bb9c-4dd5-8a1d-6c0b0b8ed3f3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:37:14.329979 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9b60623-bb9c-4dd5-8a1d-6c0b0b8ed3f3 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:37:14.330741 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-e53a135c-9da0-4549-b4c3-5b9a316927ce admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '316fd0c7342946638423ecfc7a506dfb'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:37:14.332128 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-e53a135c-9da0-4549-b4c3-5b9a316927ce admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '316fd0c7342946638423ecfc7a506dfb'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:37:14.338388 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:37:14.353979 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-e53a135c-9da0-4549-b4c3-5b9a316927ce admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '316fd0c7342946638423ecfc7a506dfb'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:37:14.371335 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 829/1675] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jul 25 05:37:14 2022] GET /identity/v3/roles => generated 1420 bytes in 102 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 05:37:14.372725 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4943a21-f320-44ee-84d7-aab88c7020f2 None None] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:37:14.383034 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4943a21-f320-44ee-84d7-aab88c7020f2 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:37:14.383350 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4943a21-f320-44ee-84d7-aab88c7020f2 None None] PATH_INFO: `/` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:37:14.388915 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 830/1676] 10.222.0.33 () {54 vars in 921 bytes} [Mon Jul 25 05:37:14 2022] GET /identity => generated 270 bytes in 17 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:37:14.389772 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4943a21-f320-44ee-84d7-aab88c7020f2 None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:37:14.390727 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-e53a135c-9da0-4549-b4c3-5b9a316927ce admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '316fd0c7342946638423ecfc7a506dfb'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:37:14.392900 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-e53a135c-9da0-4549-b4c3-5b9a316927ce admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '316fd0c7342946638423ecfc7a506dfb'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:37:14.401741 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-b4943a21-f320-44ee-84d7-aab88c7020f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:37:14.430113 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 847/1677] 10.222.0.33 () {58 vars in 1223 bytes} [Mon Jul 25 05:37:14 2022] DELETE /identity/v3/users/316fd0c7342946638423ecfc7a506dfb => generated 0 bytes in 326 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 05:37:14.436949 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e449788e-2636-4487-9c80-ba06079cff16 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:37:14.437372 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e449788e-2636-4487-9c80-ba06079cff16 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:37:14.437533 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e449788e-2636-4487-9c80-ba06079cff16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:37:14.439189 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4943a21-f320-44ee-84d7-aab88c7020f2 None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:37:14.523938 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-e449788e-2636-4487-9c80-ba06079cff16 None None] MFA Rules not processed for user `1eae540242d94c10b0ad33960a855f08`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:37:14.528226 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-e449788e-2636-4487-9c80-ba06079cff16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:37:14.581445 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-940d3a47-1d74-4231-92c0-25cb25e97ee2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:37:14.582660 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940d3a47-1d74-4231-92c0-25cb25e97ee2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:37:14.583609 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940d3a47-1d74-4231-92c0-25cb25e97ee2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:37:14.584031 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940d3a47-1d74-4231-92c0-25cb25e97ee2 admin admin] PATH_INFO: `/v3/projects/e6f640be419b4c05b9b614d5b12b1fda/users/9e1e85962574404f9e5d67f96f928e85/roles/5074e74e35f54d4b9bb415c34999a82a` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:37:14.642906 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-940d3a47-1d74-4231-92c0-25cb25e97ee2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e6f640be419b4c05b9b614d5b12b1fda, user_id=9e1e85962574404f9e5d67f96f928e85, role_id=5074e74e35f54d4b9bb415c34999a82a)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:37:14.647399 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-940d3a47-1d74-4231-92c0-25cb25e97ee2 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:37:14.692371 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 831/1678] 10.222.0.33 () {60 vars in 1482 bytes} [Mon Jul 25 05:37:14 2022] PUT /identity/v3/projects/e6f640be419b4c05b9b614d5b12b1fda/users/9e1e85962574404f9e5d67f96f928e85/roles/5074e74e35f54d4b9bb415c34999a82a => generated 0 bytes in 303 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 05:37:14.694084 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc37dff-194b-462c-9819-c4d1c8241fdb None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:37:14.694579 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc37dff-194b-462c-9819-c4d1c8241fdb None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:37:14.695095 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc37dff-194b-462c-9819-c4d1c8241fdb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:37:14.771744 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-dcc37dff-194b-462c-9819-c4d1c8241fdb None None] MFA Rules not processed for user `317790016e7047fe96af73925de5d872`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:37:14.778841 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-dcc37dff-194b-462c-9819-c4d1c8241fdb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:37:14.797432 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 848/1679] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 25 05:37:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 363 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:37:14.814459 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d834aa8e-2433-41cc-b117-2b3d8e663fd0 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:37:14.814917 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d834aa8e-2433-41cc-b117-2b3d8e663fd0 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:37:14.815403 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d834aa8e-2433-41cc-b117-2b3d8e663fd0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:37:14.901915 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-d834aa8e-2433-41cc-b117-2b3d8e663fd0 None None] MFA Rules not processed for user `9e1e85962574404f9e5d67f96f928e85`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:37:14.905199 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-d834aa8e-2433-41cc-b117-2b3d8e663fd0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:37:15.034761 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 832/1680] 10.222.0.33 () {58 vars in 1018 bytes} [Mon Jul 25 05:37:14 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 341 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:37:15.035662 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc37dff-194b-462c-9819-c4d1c8241fdb None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:37:15.043821 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-dcc37dff-194b-462c-9819-c4d1c8241fdb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:37:15.111682 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 849/1681] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 25 05:37:14 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 314 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:37:15.118096 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d834aa8e-2433-41cc-b117-2b3d8e663fd0 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:37:15.137276 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:37:15.233055 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc37dff-194b-462c-9819-c4d1c8241fdb None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:37:15.247232 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122548f6-b15d-42b7-9d6e-3a0567948b96 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b2931200f99e43d79a50a9f6e02738e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92afba266cce4d1bad4134c351e74be3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:37:15.248368 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122548f6-b15d-42b7-9d6e-3a0567948b96 service neutron] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:37:15.249451 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122548f6-b15d-42b7-9d6e-3a0567948b96 service neutron] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:37:15.250050 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122548f6-b15d-42b7-9d6e-3a0567948b96 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:37:15.308895 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d834aa8e-2433-41cc-b117-2b3d8e663fd0 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:37:15.315977 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ddf655c-98dc-4c6d-b7af-00e443ccc551 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:37:15.319634 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ddf655c-98dc-4c6d-b7af-00e443ccc551 admin admin] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:37:15.319993 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ddf655c-98dc-4c6d-b7af-00e443ccc551 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:37:15.320305 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ddf655c-98dc-4c6d-b7af-00e443ccc551 admin admin] PATH_INFO: `/v3/projects` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:37:15.321206 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ddf655c-98dc-4c6d-b7af-00e443ccc551 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:37:15.324803 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ddf655c-98dc-4c6d-b7af-00e443ccc551 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:37:15.326606 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:37:15.378719 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 850/1682] 10.222.0.33 () {60 vars in 1152 bytes} [Mon Jul 25 05:37:15 2022] POST /identity/v3/projects => generated 381 bytes in 261 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 05:37:15.379979 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ddf655c-98dc-4c6d-b7af-00e443ccc551 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:37:15.384464 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-2ddf655c-98dc-4c6d-b7af-00e443ccc551 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:37:15.395496 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-122548f6-b15d-42b7-9d6e-3a0567948b96 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:37:15.397536 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-122548f6-b15d-42b7-9d6e-3a0567948b96 service neutron] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:37:15.424710 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 833/1683] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jul 25 05:37:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 389 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 05:37:15.425869 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122548f6-b15d-42b7-9d6e-3a0567948b96 service neutron] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:37:15.468021 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122548f6-b15d-42b7-9d6e-3a0567948b96 service neutron] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:37:15.580998 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c47080a-bd11-4256-8edb-0508106492b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:37:15.581687 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c47080a-bd11-4256-8edb-0508106492b2 admin admin] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:37:15.581975 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c47080a-bd11-4256-8edb-0508106492b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:37:15.582246 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c47080a-bd11-4256-8edb-0508106492b2 admin admin] PATH_INFO: `/v3/users` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:37:15.583136 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c47080a-bd11-4256-8edb-0508106492b2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:37:15.585479 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c47080a-bd11-4256-8edb-0508106492b2 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:37:15.598272 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ddf655c-98dc-4c6d-b7af-00e443ccc551 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:37:15.608717 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eee0992-4ad0-46ea-a36f-bb16ecea3f54 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '317790016e7047fe96af73925de5d872', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92afba266cce4d1bad4134c351e74be3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:37:15.609402 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eee0992-4ad0-46ea-a36f-bb16ecea3f54 service placement] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:37:15.609682 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eee0992-4ad0-46ea-a36f-bb16ecea3f54 service placement] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:37:15.610011 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eee0992-4ad0-46ea-a36f-bb16ecea3f54 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:37:15.627763 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eee0992-4ad0-46ea-a36f-bb16ecea3f54 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:37:15.627947 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 834/1684] 10.222.0.33 () {60 vars in 1143 bytes} [Mon Jul 25 05:37:15 2022] POST /identity/v3/users => generated 297 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 05:37:15.629375 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c47080a-bd11-4256-8edb-0508106492b2 admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:37:15.629551 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eee0992-4ad0-46ea-a36f-bb16ecea3f54 service placement] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:37:15.642398 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c47080a-bd11-4256-8edb-0508106492b2 admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:37:15.654171 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 851/1685] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jul 25 05:37:15 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 274 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 05:37:15.673512 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37037824-76ac-4fe0-b29f-f6439d52abed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:37:15.677040 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37037824-76ac-4fe0-b29f-f6439d52abed admin admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:37:15.677040 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37037824-76ac-4fe0-b29f-f6439d52abed admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:37:15.679210 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37037824-76ac-4fe0-b29f-f6439d52abed admin admin] PATH_INFO: `/v3/roles` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:37:15.680462 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37037824-76ac-4fe0-b29f-f6439d52abed admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:37:15.683387 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37037824-76ac-4fe0-b29f-f6439d52abed admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:37:15.700660 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 835/1686] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jul 25 05:37:15 2022] GET /identity/v3/roles => generated 1420 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 05:37:15.700975 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eee0992-4ad0-46ea-a36f-bb16ecea3f54 service placement] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:37:15.713054 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eee0992-4ad0-46ea-a36f-bb16ecea3f54 service placement] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:37:15.724332 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e5a599-4b11-4366-9b4e-5e6cd1190938 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:37:15.725384 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e5a599-4b11-4366-9b4e-5e6cd1190938 admin admin] REQUEST_METHOD: `PUT` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:37:15.725672 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e5a599-4b11-4366-9b4e-5e6cd1190938 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:37:15.726344 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e5a599-4b11-4366-9b4e-5e6cd1190938 admin admin] PATH_INFO: `/v3/projects/2b382be51c0a41d38d8b345038ed3daf/users/f4686cad93e14779b611e747b458c0f2/roles/39e5c666285a42fe8fc2c3afca6260d1` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:37:15.764287 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3e5a599-4b11-4366-9b4e-5e6cd1190938 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2b382be51c0a41d38d8b345038ed3daf, user_id=f4686cad93e14779b611e747b458c0f2, role_id=39e5c666285a42fe8fc2c3afca6260d1)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:37:15.778476 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3e5a599-4b11-4366-9b4e-5e6cd1190938 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:37:15.812951 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 852/1687] 10.222.0.33 () {60 vars in 1482 bytes} [Mon Jul 25 05:37:15 2022] PUT /identity/v3/projects/2b382be51c0a41d38d8b345038ed3daf/users/f4686cad93e14779b611e747b458c0f2/roles/39e5c666285a42fe8fc2c3afca6260d1 => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 05:37:15.817252 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37037824-76ac-4fe0-b29f-f6439d52abed admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:37:15.830625 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37037824-76ac-4fe0-b29f-f6439d52abed admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:37:15.941754 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c72a8605-c9b5-4925-9c3e-6048f974ba74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:37:15.942639 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72a8605-c9b5-4925-9c3e-6048f974ba74 admin admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:37:15.943512 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72a8605-c9b5-4925-9c3e-6048f974ba74 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:37:15.944026 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72a8605-c9b5-4925-9c3e-6048f974ba74 admin admin] PATH_INFO: `/v3/roles` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:37:15.945252 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c72a8605-c9b5-4925-9c3e-6048f974ba74 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:37:15.947128 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c72a8605-c9b5-4925-9c3e-6048f974ba74 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:37:15.971247 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 836/1688] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jul 25 05:37:15 2022] GET /identity/v3/roles => generated 1420 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 25 05:37:15.971437 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e5a599-4b11-4366-9b4e-5e6cd1190938 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:37:15.983770 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e5a599-4b11-4366-9b4e-5e6cd1190938 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:37:15.988712 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7bb790b-a7a9-4d9b-8624-12c3e4d6c3e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:37:15.989689 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7bb790b-a7a9-4d9b-8624-12c3e4d6c3e3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:37:15.989975 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7bb790b-a7a9-4d9b-8624-12c3e4d6c3e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:37:15.990249 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7bb790b-a7a9-4d9b-8624-12c3e4d6c3e3 admin admin] PATH_INFO: `/v3/projects/2b382be51c0a41d38d8b345038ed3daf/users/f4686cad93e14779b611e747b458c0f2/roles/39e5c666285a42fe8fc2c3afca6260d1` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:37:15.992273 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7bb790b-a7a9-4d9b-8624-12c3e4d6c3e3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2b382be51c0a41d38d8b345038ed3daf, user_id=f4686cad93e14779b611e747b458c0f2, role_id=39e5c666285a42fe8fc2c3afca6260d1)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:37:15.994987 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7bb790b-a7a9-4d9b-8624-12c3e4d6c3e3 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:37:16.019715 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 853/1689] 10.222.0.33 () {60 vars in 1482 bytes} [Mon Jul 25 05:37:15 2022] PUT /identity/v3/projects/2b382be51c0a41d38d8b345038ed3daf/users/f4686cad93e14779b611e747b458c0f2/roles/39e5c666285a42fe8fc2c3afca6260d1 => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 05:37:16.020733 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5071fcbb-cb9a-46aa-9ee8-de2444068620 None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:37:16.021077 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5071fcbb-cb9a-46aa-9ee8-de2444068620 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:37:16.021366 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5071fcbb-cb9a-46aa-9ee8-de2444068620 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:37:16.062742 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-5071fcbb-cb9a-46aa-9ee8-de2444068620 None None] MFA Rules not processed for user `f4686cad93e14779b611e747b458c0f2`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:37:16.065933 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-5071fcbb-cb9a-46aa-9ee8-de2444068620 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:37:16.132718 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:37:16.213217 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 837/1690] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 25 05:37:16 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:37:16.227431 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7bb790b-a7a9-4d9b-8624-12c3e4d6c3e3 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:37:16.230240 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-b7bb790b-a7a9-4d9b-8624-12c3e4d6c3e3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:37:16.332587 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:37:16.346028 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7bb790b-a7a9-4d9b-8624-12c3e4d6c3e3 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:37:16.371279 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91648fb9-3e9f-42f1-b02b-8936f3992f68 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db57b478c25643dbaf936f90f6e17d0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92afba266cce4d1bad4134c351e74be3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:37:16.371279 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91648fb9-3e9f-42f1-b02b-8936f3992f68 service cinder] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:37:16.371279 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91648fb9-3e9f-42f1-b02b-8936f3992f68 service cinder] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:37:16.371279 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91648fb9-3e9f-42f1-b02b-8936f3992f68 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:37:16.392562 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91648fb9-3e9f-42f1-b02b-8936f3992f68 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:37:16.393933 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91648fb9-3e9f-42f1-b02b-8936f3992f68 service cinder] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:37:16.410269 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 854/1691] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jul 25 05:37:16 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 183 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 05:37:16.955469 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02fa39c0-8681-49da-b562-89ed056fec9b None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:37:16.955988 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02fa39c0-8681-49da-b562-89ed056fec9b None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:37:16.956068 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02fa39c0-8681-49da-b562-89ed056fec9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:37:17.000658 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-02fa39c0-8681-49da-b562-89ed056fec9b None None] MFA Rules not processed for user `bf61ffac0e3143f6b44971158bd7a553`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:37:17.002828 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-02fa39c0-8681-49da-b562-89ed056fec9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:37:17.123990 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 838/1692] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 25 05:37:16 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:37:17.136595 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb833b7f-0d2d-4f58-9e8a-3259a18a49c1 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:37:17.137305 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb833b7f-0d2d-4f58-9e8a-3259a18a49c1 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:37:17.137701 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb833b7f-0d2d-4f58-9e8a-3259a18a49c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:37:17.161937 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-eb833b7f-0d2d-4f58-9e8a-3259a18a49c1 None None] MFA Rules not processed for user `bf61ffac0e3143f6b44971158bd7a553`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:37:17.163647 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02fa39c0-8681-49da-b562-89ed056fec9b None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:37:17.170436 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-eb833b7f-0d2d-4f58-9e8a-3259a18a49c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:37:17.177498 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02fa39c0-8681-49da-b562-89ed056fec9b None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:37:17.187641 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-242a4b28-83b0-4943-ba9a-1a791e451931 tempest-VolumeQuotaClassesTest-1324915465 tempest-VolumeQuotaClassesTest-1324915465-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4686cad93e14779b611e747b458c0f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b382be51c0a41d38d8b345038ed3daf', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:37:17.188648 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242a4b28-83b0-4943-ba9a-1a791e451931 tempest-VolumeQuotaClassesTest-1324915465 tempest-VolumeQuotaClassesTest-1324915465-project-admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:37:17.189659 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242a4b28-83b0-4943-ba9a-1a791e451931 tempest-VolumeQuotaClassesTest-1324915465 tempest-VolumeQuotaClassesTest-1324915465-project-admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:37:17.191115 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242a4b28-83b0-4943-ba9a-1a791e451931 tempest-VolumeQuotaClassesTest-1324915465 tempest-VolumeQuotaClassesTest-1324915465-project-admin] PATH_INFO: `/v3/domains` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:37:17.192751 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 855/1693] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 25 05:37:17 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 60 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:37:17.193086 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-242a4b28-83b0-4943-ba9a-1a791e451931 tempest-VolumeQuotaClassesTest-1324915465 tempest-VolumeQuotaClassesTest-1324915465-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:37:17.193519 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-242a4b28-83b0-4943-ba9a-1a791e451931 tempest-VolumeQuotaClassesTest-1324915465 tempest-VolumeQuotaClassesTest-1324915465-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:37:17.196716 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-242a4b28-83b0-4943-ba9a-1a791e451931 tempest-VolumeQuotaClassesTest-1324915465 tempest-VolumeQuotaClassesTest-1324915465-project-admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:37:17.207713 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb833b7f-0d2d-4f58-9e8a-3259a18a49c1 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:37:17.209625 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-eb833b7f-0d2d-4f58-9e8a-3259a18a49c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:37:17.226885 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 839/1694] 10.222.0.33 () {58 vars in 1152 bytes} [Mon Jul 25 05:37:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:37:17.232620 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-242a4b28-83b0-4943-ba9a-1a791e451931 tempest-VolumeQuotaClassesTest-1324915465 tempest-VolumeQuotaClassesTest-1324915465-project-admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:37:17.258600 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-242a4b28-83b0-4943-ba9a-1a791e451931 tempest-VolumeQuotaClassesTest-1324915465 tempest-VolumeQuotaClassesTest-1324915465-project-admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:37:17.271400 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ad8938-d281-4b43-a860-074af07e1092 tempest-VolumeQuotaClassesTest-1324915465 tempest-VolumeQuotaClassesTest-1324915465-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4686cad93e14779b611e747b458c0f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b382be51c0a41d38d8b345038ed3daf', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:37:17.272215 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ad8938-d281-4b43-a860-074af07e1092 tempest-VolumeQuotaClassesTest-1324915465 tempest-VolumeQuotaClassesTest-1324915465-project-admin] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:37:17.272622 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ad8938-d281-4b43-a860-074af07e1092 tempest-VolumeQuotaClassesTest-1324915465 tempest-VolumeQuotaClassesTest-1324915465-project-admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:37:17.273046 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ad8938-d281-4b43-a860-074af07e1092 tempest-VolumeQuotaClassesTest-1324915465 tempest-VolumeQuotaClassesTest-1324915465-project-admin] PATH_INFO: `/v3/projects` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:37:17.275949 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28ad8938-d281-4b43-a860-074af07e1092 tempest-VolumeQuotaClassesTest-1324915465 tempest-VolumeQuotaClassesTest-1324915465-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:37:17.278616 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28ad8938-d281-4b43-a860-074af07e1092 tempest-VolumeQuotaClassesTest-1324915465 tempest-VolumeQuotaClassesTest-1324915465-project-admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:37:17.336915 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 840/1695] 10.222.0.33 () {60 vars in 1152 bytes} [Mon Jul 25 05:37:17 2022] POST /identity/v3/projects => generated 352 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 05:37:17.341847 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ad8938-d281-4b43-a860-074af07e1092 tempest-VolumeQuotaClassesTest-1324915465 tempest-VolumeQuotaClassesTest-1324915465-project-admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:37:17.374105 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb833b7f-0d2d-4f58-9e8a-3259a18a49c1 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:37:17.447640 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ad8938-d281-4b43-a860-074af07e1092 tempest-VolumeQuotaClassesTest-1324915465 tempest-VolumeQuotaClassesTest-1324915465-project-admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:37:17.471884 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecbacc0b-18a8-4773-90d5-fe940031533d tempest-VolumeQuotaClassesTest-1324915465 tempest-VolumeQuotaClassesTest-1324915465-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4686cad93e14779b611e747b458c0f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b382be51c0a41d38d8b345038ed3daf', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:37:17.472364 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecbacc0b-18a8-4773-90d5-fe940031533d tempest-VolumeQuotaClassesTest-1324915465 tempest-VolumeQuotaClassesTest-1324915465-project-admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:37:17.472654 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecbacc0b-18a8-4773-90d5-fe940031533d tempest-VolumeQuotaClassesTest-1324915465 tempest-VolumeQuotaClassesTest-1324915465-project-admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:37:17.473221 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecbacc0b-18a8-4773-90d5-fe940031533d tempest-VolumeQuotaClassesTest-1324915465 tempest-VolumeQuotaClassesTest-1324915465-project-admin] PATH_INFO: `/v3/domains` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:37:17.475512 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecbacc0b-18a8-4773-90d5-fe940031533d tempest-VolumeQuotaClassesTest-1324915465 tempest-VolumeQuotaClassesTest-1324915465-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:37:17.475831 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecbacc0b-18a8-4773-90d5-fe940031533d tempest-VolumeQuotaClassesTest-1324915465 tempest-VolumeQuotaClassesTest-1324915465-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:37:17.477508 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecbacc0b-18a8-4773-90d5-fe940031533d tempest-VolumeQuotaClassesTest-1324915465 tempest-VolumeQuotaClassesTest-1324915465-project-admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:37:17.519297 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 841/1696] 10.222.0.33 () {58 vars in 1152 bytes} [Mon Jul 25 05:37:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 05:37:17.521250 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecbacc0b-18a8-4773-90d5-fe940031533d tempest-VolumeQuotaClassesTest-1324915465 tempest-VolumeQuotaClassesTest-1324915465-project-admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:37:17.527021 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-ecbacc0b-18a8-4773-90d5-fe940031533d tempest-VolumeQuotaClassesTest-1324915465 tempest-VolumeQuotaClassesTest-1324915465-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:37:17.532904 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3684b9bf-47f1-4804-9b92-3062c1fa32b9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cd482a61e8a420db45bb532e5dae6e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92afba266cce4d1bad4134c351e74be3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:37:17.533407 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3684b9bf-47f1-4804-9b92-3062c1fa32b9 service nova] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:37:17.534181 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3684b9bf-47f1-4804-9b92-3062c1fa32b9 service nova] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:37:17.534181 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3684b9bf-47f1-4804-9b92-3062c1fa32b9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:37:17.561608 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:37:17.679255 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3684b9bf-47f1-4804-9b92-3062c1fa32b9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:37:17.681761 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3684b9bf-47f1-4804-9b92-3062c1fa32b9 service nova] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:37:17.704871 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 856/1697] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jul 25 05:37:17 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 497 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 05:37:17.705852 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3684b9bf-47f1-4804-9b92-3062c1fa32b9 service nova] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:37:17.723980 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecbacc0b-18a8-4773-90d5-fe940031533d tempest-VolumeQuotaClassesTest-1324915465 tempest-VolumeQuotaClassesTest-1324915465-project-admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:37:17.738281 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3684b9bf-47f1-4804-9b92-3062c1fa32b9 service nova] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:37:17.754198 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21206a01-d248-45cf-95f1-15e20598dd3f tempest-VolumeQuotaClassesTest-1324915465 tempest-VolumeQuotaClassesTest-1324915465-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4686cad93e14779b611e747b458c0f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b382be51c0a41d38d8b345038ed3daf', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:37:17.756127 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be98d14-862f-4ebf-91c3-ebc1a3b29462 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:37:17.758089 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21206a01-d248-45cf-95f1-15e20598dd3f tempest-VolumeQuotaClassesTest-1324915465 tempest-VolumeQuotaClassesTest-1324915465-project-admin] REQUEST_METHOD: `DELETE` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:37:17.759813 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be98d14-862f-4ebf-91c3-ebc1a3b29462 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:37:17.760297 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be98d14-862f-4ebf-91c3-ebc1a3b29462 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:37:17.760660 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21206a01-d248-45cf-95f1-15e20598dd3f tempest-VolumeQuotaClassesTest-1324915465 tempest-VolumeQuotaClassesTest-1324915465-project-admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:37:17.761079 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be98d14-862f-4ebf-91c3-ebc1a3b29462 admin admin] PATH_INFO: `/v3/projects/129b404cb6484de8962c1aa49e34cca1` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:37:17.761252 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21206a01-d248-45cf-95f1-15e20598dd3f tempest-VolumeQuotaClassesTest-1324915465 tempest-VolumeQuotaClassesTest-1324915465-project-admin] PATH_INFO: `/v3/projects/f89d5b01a4744d97ba85086d56bed578` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:37:17.763475 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21206a01-d248-45cf-95f1-15e20598dd3f tempest-VolumeQuotaClassesTest-1324915465 tempest-VolumeQuotaClassesTest-1324915465-project-admin] RBAC: Authorizing `identity:delete_project(project_id=f89d5b01a4744d97ba85086d56bed578)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:37:17.765880 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6be98d14-862f-4ebf-91c3-ebc1a3b29462 admin admin] RBAC: Authorizing `identity:delete_project(project_id=129b404cb6484de8962c1aa49e34cca1)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:37:17.769077 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21206a01-d248-45cf-95f1-15e20598dd3f tempest-VolumeQuotaClassesTest-1324915465 tempest-VolumeQuotaClassesTest-1324915465-project-admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:37:17.773684 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6be98d14-862f-4ebf-91c3-ebc1a3b29462 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:37:17.821418 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:37:17.892215 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-21206a01-d248-45cf-95f1-15e20598dd3f tempest-VolumeQuotaClassesTest-1324915465 tempest-VolumeQuotaClassesTest-1324915465-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f89d5b01a4744d97ba85086d56bed578', 'request_id': 'req-21206a01-d248-45cf-95f1-15e20598dd3f'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:37:17.915660 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-21206a01-d248-45cf-95f1-15e20598dd3f tempest-VolumeQuotaClassesTest-1324915465 tempest-VolumeQuotaClassesTest-1324915465-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f89d5b01a4744d97ba85086d56bed578', 'request_id': 'req-21206a01-d248-45cf-95f1-15e20598dd3f'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:37:17.937756 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-6be98d14-862f-4ebf-91c3-ebc1a3b29462 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '129b404cb6484de8962c1aa49e34cca1', 'request_id': 'req-6be98d14-862f-4ebf-91c3-ebc1a3b29462'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:37:17.941410 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-21206a01-d248-45cf-95f1-15e20598dd3f tempest-VolumeQuotaClassesTest-1324915465 tempest-VolumeQuotaClassesTest-1324915465-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f89d5b01a4744d97ba85086d56bed578', 'request_id': 'req-21206a01-d248-45cf-95f1-15e20598dd3f'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:37:17.956385 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-6be98d14-862f-4ebf-91c3-ebc1a3b29462 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '129b404cb6484de8962c1aa49e34cca1', 'request_id': 'req-6be98d14-862f-4ebf-91c3-ebc1a3b29462'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:37:17.969260 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-21206a01-d248-45cf-95f1-15e20598dd3f tempest-VolumeQuotaClassesTest-1324915465 tempest-VolumeQuotaClassesTest-1324915465-project-admin] The token cache is being invalidate because project f89d5b01a4744d97ba85086d56bed578 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80249) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 05:37:17.969556 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-21206a01-d248-45cf-95f1-15e20598dd3f tempest-VolumeQuotaClassesTest-1324915465 tempest-VolumeQuotaClassesTest-1324915465-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:37:17.970990 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-21206a01-d248-45cf-95f1-15e20598dd3f tempest-VolumeQuotaClassesTest-1324915465 tempest-VolumeQuotaClassesTest-1324915465-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:37:17.977169 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 857/1698] 10.222.0.33 () {58 vars in 1232 bytes} [Mon Jul 25 05:37:17 2022] DELETE /identity/v3/projects/f89d5b01a4744d97ba85086d56bed578 => generated 0 bytes in 268 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 05:37:17.978660 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18b27ac-5e4f-4ea2-bbeb-2da79ecd2086 None None] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:37:17.978960 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18b27ac-5e4f-4ea2-bbeb-2da79ecd2086 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:37:17.979243 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18b27ac-5e4f-4ea2-bbeb-2da79ecd2086 None None] PATH_INFO: `/` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:37:17.984929 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-6be98d14-862f-4ebf-91c3-ebc1a3b29462 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '129b404cb6484de8962c1aa49e34cca1', 'request_id': 'req-6be98d14-862f-4ebf-91c3-ebc1a3b29462'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:37:17.992143 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 858/1699] 10.222.0.33 () {54 vars in 921 bytes} [Mon Jul 25 05:37:17 2022] GET /identity => generated 270 bytes in 14 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:37:18.023138 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ca7239-7d8d-49f7-ab66-322543ec517d None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:37:18.023435 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ca7239-7d8d-49f7-ab66-322543ec517d None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:37:18.023720 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ca7239-7d8d-49f7-ab66-322543ec517d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:37:18.027350 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-6be98d14-862f-4ebf-91c3-ebc1a3b29462 admin admin] The token cache is being invalidate because project 129b404cb6484de8962c1aa49e34cca1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80248) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 05:37:18.027645 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-6be98d14-862f-4ebf-91c3-ebc1a3b29462 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:37:18.029864 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-6be98d14-862f-4ebf-91c3-ebc1a3b29462 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:37:18.036907 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 842/1700] 10.222.0.33 () {58 vars in 1232 bytes} [Mon Jul 25 05:37:17 2022] DELETE /identity/v3/projects/129b404cb6484de8962c1aa49e34cca1 => generated 0 bytes in 517 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 05:37:18.053796 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1f133c-6303-4d9f-a3c8-bd7ab3665a9b None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:37:18.054324 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1f133c-6303-4d9f-a3c8-bd7ab3665a9b None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:37:18.054747 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1f133c-6303-4d9f-a3c8-bd7ab3665a9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:37:18.072665 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-70ca7239-7d8d-49f7-ab66-322543ec517d None None] MFA Rules not processed for user `317790016e7047fe96af73925de5d872`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:37:18.075159 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-70ca7239-7d8d-49f7-ab66-322543ec517d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:37:18.110015 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-1f1f133c-6303-4d9f-a3c8-bd7ab3665a9b None None] MFA Rules not processed for user `1eae540242d94c10b0ad33960a855f08`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:37:18.111756 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-1f1f133c-6303-4d9f-a3c8-bd7ab3665a9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:37:18.251510 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 859/1701] 10.222.0.33 () {58 vars in 1018 bytes} [Mon Jul 25 05:37:18 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 229 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 05:37:18.310283 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ca7239-7d8d-49f7-ab66-322543ec517d None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:37:18.311775 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-70ca7239-7d8d-49f7-ab66-322543ec517d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:37:18.319178 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 843/1702] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 25 05:37:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 275 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:37:18.321470 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b361ce4e-8ce6-4496-a6be-6b52879f0972 None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:37:18.321764 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b361ce4e-8ce6-4496-a6be-6b52879f0972 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:37:18.322046 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b361ce4e-8ce6-4496-a6be-6b52879f0972 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:37:18.340555 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ca7239-7d8d-49f7-ab66-322543ec517d None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:37:18.382940 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e2c26d1-2142-4f77-ab4b-65a32346dcd5 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '317790016e7047fe96af73925de5d872', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92afba266cce4d1bad4134c351e74be3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:37:18.384815 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-b361ce4e-8ce6-4496-a6be-6b52879f0972 None None] MFA Rules not processed for user `1eae540242d94c10b0ad33960a855f08`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:37:18.386264 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-b361ce4e-8ce6-4496-a6be-6b52879f0972 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:37:18.390883 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2c26d1-2142-4f77-ab4b-65a32346dcd5 service placement] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:37:18.391405 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2c26d1-2142-4f77-ab4b-65a32346dcd5 service placement] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:37:18.392564 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2c26d1-2142-4f77-ab4b-65a32346dcd5 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:37:18.437993 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 844/1703] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 25 05:37:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:37:18.442402 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b361ce4e-8ce6-4496-a6be-6b52879f0972 None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:37:18.445750 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-b361ce4e-8ce6-4496-a6be-6b52879f0972 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:37:18.462634 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e2c26d1-2142-4f77-ab4b-65a32346dcd5 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:37:18.474051 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e2c26d1-2142-4f77-ab4b-65a32346dcd5 service placement] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:37:18.486317 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b361ce4e-8ce6-4496-a6be-6b52879f0972 None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:37:18.492519 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8589351-841e-4594-9818-9ce76b62f620 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:37:18.501085 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8589351-841e-4594-9818-9ce76b62f620 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:37:18.501604 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8589351-841e-4594-9818-9ce76b62f620 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:37:18.502158 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8589351-841e-4594-9818-9ce76b62f620 admin admin] PATH_INFO: `/v3/users/aec7125a032f42d19b106b79c9968d46` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:37:18.503783 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8589351-841e-4594-9818-9ce76b62f620 admin admin] RBAC: Authorizing `identity:delete_user(user_id=aec7125a032f42d19b106b79c9968d46)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:37:18.506034 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8589351-841e-4594-9818-9ce76b62f620 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:37:18.512588 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 860/1704] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jul 25 05:37:18 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 203 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 25 05:37:18.513593 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e2c26d1-2142-4f77-ab4b-65a32346dcd5 service placement] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:37:18.552731 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e2c26d1-2142-4f77-ab4b-65a32346dcd5 service placement] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:37:18.571900 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d89297f9-ef94-4784-a36e-732db8cf8ebb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:37:18.572643 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89297f9-ef94-4784-a36e-732db8cf8ebb admin admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:37:18.572965 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89297f9-ef94-4784-a36e-732db8cf8ebb admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:37:18.573748 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89297f9-ef94-4784-a36e-732db8cf8ebb admin admin] PATH_INFO: `/v3/domains` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:37:18.574673 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d89297f9-ef94-4784-a36e-732db8cf8ebb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:37:18.574995 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d89297f9-ef94-4784-a36e-732db8cf8ebb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:37:18.583140 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d89297f9-ef94-4784-a36e-732db8cf8ebb admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:37:18.615875 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:37:18.649284 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 861/1705] 10.222.0.33 () {58 vars in 1152 bytes} [Mon Jul 25 05:37:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:37:18.649284 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d89297f9-ef94-4784-a36e-732db8cf8ebb admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:37:18.650901 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-d89297f9-ef94-4784-a36e-732db8cf8ebb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:37:18.672909 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-d8589351-841e-4594-9818-9ce76b62f620 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'aec7125a032f42d19b106b79c9968d46'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:37:18.681641 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d89297f9-ef94-4784-a36e-732db8cf8ebb admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:37:18.695017 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b466b8ec-3b85-44e9-a82e-624616297691 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:37:18.695918 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b466b8ec-3b85-44e9-a82e-624616297691 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:37:18.696217 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b466b8ec-3b85-44e9-a82e-624616297691 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:37:18.696487 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b466b8ec-3b85-44e9-a82e-624616297691 admin admin] PATH_INFO: `/v3/users/da3b4bd3f89842c7af66f9761ba87116` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:37:18.697957 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b466b8ec-3b85-44e9-a82e-624616297691 admin admin] RBAC: Authorizing `identity:delete_user(user_id=da3b4bd3f89842c7af66f9761ba87116)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:37:18.702149 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b466b8ec-3b85-44e9-a82e-624616297691 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:37:18.707976 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-d8589351-841e-4594-9818-9ce76b62f620 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'aec7125a032f42d19b106b79c9968d46'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:37:18.708849 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-d8589351-841e-4594-9818-9ce76b62f620 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'aec7125a032f42d19b106b79c9968d46'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:37:18.732461 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-d8589351-841e-4594-9818-9ce76b62f620 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'aec7125a032f42d19b106b79c9968d46'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:37:18.733131 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-d8589351-841e-4594-9818-9ce76b62f620 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'aec7125a032f42d19b106b79c9968d46'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:37:18.755899 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 845/1706] 10.222.0.33 () {58 vars in 1223 bytes} [Mon Jul 25 05:37:18 2022] DELETE /identity/v3/users/aec7125a032f42d19b106b79c9968d46 => generated 0 bytes in 317 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 05:37:18.756750 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8589351-841e-4594-9818-9ce76b62f620 admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:37:18.764840 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-d8589351-841e-4594-9818-9ce76b62f620 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:37:18.826336 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-b466b8ec-3b85-44e9-a82e-624616297691 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'da3b4bd3f89842c7af66f9761ba87116'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:37:18.835455 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-b466b8ec-3b85-44e9-a82e-624616297691 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'da3b4bd3f89842c7af66f9761ba87116'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:37:18.857451 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-b466b8ec-3b85-44e9-a82e-624616297691 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'da3b4bd3f89842c7af66f9761ba87116'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:37:18.878043 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:37:18.891603 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-b466b8ec-3b85-44e9-a82e-624616297691 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'da3b4bd3f89842c7af66f9761ba87116'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:37:18.892422 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-b466b8ec-3b85-44e9-a82e-624616297691 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'da3b4bd3f89842c7af66f9761ba87116'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:37:18.925785 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 862/1707] 10.222.0.33 () {58 vars in 1223 bytes} [Mon Jul 25 05:37:18 2022] DELETE /identity/v3/users/da3b4bd3f89842c7af66f9761ba87116 => generated 0 bytes in 279 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 05:37:18.929868 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b466b8ec-3b85-44e9-a82e-624616297691 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:37:18.933920 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-b466b8ec-3b85-44e9-a82e-624616297691 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:37:18.971030 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8589351-841e-4594-9818-9ce76b62f620 admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:37:19.087270 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba67595-0051-4b54-a28b-ad0eaba8ee84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:37:19.088384 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba67595-0051-4b54-a28b-ad0eaba8ee84 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:37:19.088950 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba67595-0051-4b54-a28b-ad0eaba8ee84 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:37:19.090417 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba67595-0051-4b54-a28b-ad0eaba8ee84 admin admin] PATH_INFO: `/v3/users/9e1e85962574404f9e5d67f96f928e85` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:37:19.092840 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ba67595-0051-4b54-a28b-ad0eaba8ee84 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9e1e85962574404f9e5d67f96f928e85)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:37:19.097492 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ba67595-0051-4b54-a28b-ad0eaba8ee84 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:37:19.122026 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b466b8ec-3b85-44e9-a82e-624616297691 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:37:19.127920 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45da9b9b-e0f2-4431-a1d0-d070348e47b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:37:19.129386 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45da9b9b-e0f2-4431-a1d0-d070348e47b2 admin admin] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:37:19.130023 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45da9b9b-e0f2-4431-a1d0-d070348e47b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:37:19.130383 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45da9b9b-e0f2-4431-a1d0-d070348e47b2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:37:19.131553 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45da9b9b-e0f2-4431-a1d0-d070348e47b2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:37:19.146295 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45da9b9b-e0f2-4431-a1d0-d070348e47b2 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:37:19.216236 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-0ba67595-0051-4b54-a28b-ad0eaba8ee84 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9e1e85962574404f9e5d67f96f928e85'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:37:19.220036 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 863/1708] 10.222.0.33 () {60 vars in 1152 bytes} [Mon Jul 25 05:37:18 2022] POST /identity/v3/projects => generated 367 bytes in 294 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 05:37:19.223671 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f56e3d-23ae-4a6b-8b52-326ba903d3a9 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:37:19.224116 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f56e3d-23ae-4a6b-8b52-326ba903d3a9 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:37:19.224410 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f56e3d-23ae-4a6b-8b52-326ba903d3a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:37:19.226511 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:37:19.235349 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-0ba67595-0051-4b54-a28b-ad0eaba8ee84 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9e1e85962574404f9e5d67f96f928e85'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:37:19.237301 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-0ba67595-0051-4b54-a28b-ad0eaba8ee84 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9e1e85962574404f9e5d67f96f928e85'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:37:19.250175 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-0ba67595-0051-4b54-a28b-ad0eaba8ee84 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9e1e85962574404f9e5d67f96f928e85'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:37:19.252059 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-0ba67595-0051-4b54-a28b-ad0eaba8ee84 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9e1e85962574404f9e5d67f96f928e85'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:37:19.268631 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-31f56e3d-23ae-4a6b-8b52-326ba903d3a9 None None] MFA Rules not processed for user `1eae540242d94c10b0ad33960a855f08`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:37:19.270588 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-31f56e3d-23ae-4a6b-8b52-326ba903d3a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:37:19.302801 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 846/1709] 10.222.0.33 () {58 vars in 1223 bytes} [Mon Jul 25 05:37:18 2022] DELETE /identity/v3/users/9e1e85962574404f9e5d67f96f928e85 => generated 0 bytes in 546 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 05:37:19.308415 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba67595-0051-4b54-a28b-ad0eaba8ee84 admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:37:19.325705 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-0ba67595-0051-4b54-a28b-ad0eaba8ee84 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:37:19.475639 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 864/1710] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 25 05:37:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 254 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:37:19.476193 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d0a108-d72d-4323-9c14-7702f19803a4 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:37:19.476552 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d0a108-d72d-4323-9c14-7702f19803a4 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:37:19.476758 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d0a108-d72d-4323-9c14-7702f19803a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:37:19.525492 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-e7d0a108-d72d-4323-9c14-7702f19803a4 None None] MFA Rules not processed for user `1eae540242d94c10b0ad33960a855f08`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:37:19.527191 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-e7d0a108-d72d-4323-9c14-7702f19803a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:37:19.606347 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba67595-0051-4b54-a28b-ad0eaba8ee84 admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:37:19.629946 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f479026b-df66-46b0-b4d4-e0b1aff55a5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:37:19.631948 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f479026b-df66-46b0-b4d4-e0b1aff55a5d admin admin] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:37:19.633762 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f479026b-df66-46b0-b4d4-e0b1aff55a5d admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:37:19.635319 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f479026b-df66-46b0-b4d4-e0b1aff55a5d admin admin] PATH_INFO: `/v3/users` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:37:19.636970 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f479026b-df66-46b0-b4d4-e0b1aff55a5d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:37:19.647140 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f479026b-df66-46b0-b4d4-e0b1aff55a5d admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:37:19.710872 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 865/1711] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 25 05:37:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 235 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:37:19.713545 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d0a108-d72d-4323-9c14-7702f19803a4 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:37:19.716639 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-e7d0a108-d72d-4323-9c14-7702f19803a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:37:19.727417 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 847/1712] 10.222.0.33 () {60 vars in 1143 bytes} [Mon Jul 25 05:37:19 2022] POST /identity/v3/users => generated 284 bytes in 424 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 05:37:19.731687 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f479026b-df66-46b0-b4d4-e0b1aff55a5d admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:37:19.755675 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f479026b-df66-46b0-b4d4-e0b1aff55a5d admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:37:19.769543 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54bc0f96-07ef-4f7c-8a3f-8a2f4688198d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:37:19.770398 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54bc0f96-07ef-4f7c-8a3f-8a2f4688198d admin admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:37:19.770753 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54bc0f96-07ef-4f7c-8a3f-8a2f4688198d admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:37:19.771107 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54bc0f96-07ef-4f7c-8a3f-8a2f4688198d admin admin] PATH_INFO: `/v3/roles` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:37:19.772005 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54bc0f96-07ef-4f7c-8a3f-8a2f4688198d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:37:19.778622 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=96,keystone:DELETE=2,keystone:INSERT=2 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:37:19.786038 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54bc0f96-07ef-4f7c-8a3f-8a2f4688198d admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:37:19.813601 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 848/1713] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jul 25 05:37:19 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 05:37:19.815200 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54bc0f96-07ef-4f7c-8a3f-8a2f4688198d admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:37:19.818126 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-54bc0f96-07ef-4f7c-8a3f-8a2f4688198d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:37:19.907602 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d0a108-d72d-4323-9c14-7702f19803a4 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:37:19.920462 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ffbc8c-5d56-4f70-bad2-9446ef98e7b3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b2931200f99e43d79a50a9f6e02738e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92afba266cce4d1bad4134c351e74be3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:37:19.921266 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ffbc8c-5d56-4f70-bad2-9446ef98e7b3 service neutron] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:37:19.921556 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ffbc8c-5d56-4f70-bad2-9446ef98e7b3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:37:19.921839 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ffbc8c-5d56-4f70-bad2-9446ef98e7b3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:37:19.922734 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54bc0f96-07ef-4f7c-8a3f-8a2f4688198d admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:37:19.928808 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c69fbc-6dd5-44a7-9bd4-6aae5aba7d33 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b2931200f99e43d79a50a9f6e02738e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92afba266cce4d1bad4134c351e74be3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:37:19.930318 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c69fbc-6dd5-44a7-9bd4-6aae5aba7d33 service neutron] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:37:19.930710 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c69fbc-6dd5-44a7-9bd4-6aae5aba7d33 service neutron] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:37:19.931144 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c69fbc-6dd5-44a7-9bd4-6aae5aba7d33 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:37:19.973000 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73ffbc8c-5d56-4f70-bad2-9446ef98e7b3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:37:19.975239 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72c69fbc-6dd5-44a7-9bd4-6aae5aba7d33 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:37:19.977887 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73ffbc8c-5d56-4f70-bad2-9446ef98e7b3 service neutron] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:37:19.980491 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72c69fbc-6dd5-44a7-9bd4-6aae5aba7d33 service neutron] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:37:20.003767 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 849/1714] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jul 25 05:37:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 189 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 05:37:20.004994 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c69fbc-6dd5-44a7-9bd4-6aae5aba7d33 service neutron] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:37:20.006724 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-72c69fbc-6dd5-44a7-9bd4-6aae5aba7d33 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:37:20.018658 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 866/1715] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jul 25 05:37:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 05:37:20.019662 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ffbc8c-5d56-4f70-bad2-9446ef98e7b3 service neutron] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:37:20.048347 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ffbc8c-5d56-4f70-bad2-9446ef98e7b3 service neutron] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:37:20.076135 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c69fbc-6dd5-44a7-9bd4-6aae5aba7d33 service neutron] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:37:20.076807 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a3a9f8c-2aa8-429f-8e88-b1b7d3d15dea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:37:20.078606 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3a9f8c-2aa8-429f-8e88-b1b7d3d15dea admin admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:37:20.080320 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3a9f8c-2aa8-429f-8e88-b1b7d3d15dea admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:37:20.080714 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3a9f8c-2aa8-429f-8e88-b1b7d3d15dea admin admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:37:20.083258 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a3a9f8c-2aa8-429f-8e88-b1b7d3d15dea admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:37:20.086223 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a3a9f8c-2aa8-429f-8e88-b1b7d3d15dea admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:37:20.091057 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a28d4243-10ef-4f1e-a8cc-4ffc80b871bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:37:20.091957 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28d4243-10ef-4f1e-a8cc-4ffc80b871bd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:37:20.092254 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28d4243-10ef-4f1e-a8cc-4ffc80b871bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:37:20.092541 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28d4243-10ef-4f1e-a8cc-4ffc80b871bd admin admin] PATH_INFO: `/v3/projects/039d0c9e35dd4a74b7b08c982535155d` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:37:20.095832 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a28d4243-10ef-4f1e-a8cc-4ffc80b871bd admin admin] RBAC: Authorizing `identity:delete_project(project_id=039d0c9e35dd4a74b7b08c982535155d)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:37:20.099970 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a28d4243-10ef-4f1e-a8cc-4ffc80b871bd admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:37:20.111158 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 867/1716] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jul 25 05:37:20 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 05:37:20.112929 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3507cd-1792-4cb2-bfcb-80f00e52d235 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:37:20.113293 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3507cd-1792-4cb2-bfcb-80f00e52d235 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:37:20.113642 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3507cd-1792-4cb2-bfcb-80f00e52d235 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:37:20.195442 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-ca3507cd-1792-4cb2-bfcb-80f00e52d235 None None] MFA Rules not processed for user `3352531be1b54f76b864383ccc2e5ec2`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:37:20.196641 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-ca3507cd-1792-4cb2-bfcb-80f00e52d235 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:37:20.220159 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-a28d4243-10ef-4f1e-a8cc-4ffc80b871bd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '039d0c9e35dd4a74b7b08c982535155d', 'request_id': 'req-a28d4243-10ef-4f1e-a8cc-4ffc80b871bd'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:37:20.223799 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:37:20.229585 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-a28d4243-10ef-4f1e-a8cc-4ffc80b871bd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '039d0c9e35dd4a74b7b08c982535155d', 'request_id': 'req-a28d4243-10ef-4f1e-a8cc-4ffc80b871bd'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:37:20.252684 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-a28d4243-10ef-4f1e-a8cc-4ffc80b871bd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '039d0c9e35dd4a74b7b08c982535155d', 'request_id': 'req-a28d4243-10ef-4f1e-a8cc-4ffc80b871bd'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:37:20.282779 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-a28d4243-10ef-4f1e-a8cc-4ffc80b871bd admin admin] The token cache is being invalidate because project 039d0c9e35dd4a74b7b08c982535155d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80248) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 05:37:20.283098 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-a28d4243-10ef-4f1e-a8cc-4ffc80b871bd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:37:20.284089 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-a28d4243-10ef-4f1e-a8cc-4ffc80b871bd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:37:20.293242 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 850/1717] 10.222.0.33 () {58 vars in 1232 bytes} [Mon Jul 25 05:37:20 2022] DELETE /identity/v3/projects/039d0c9e35dd4a74b7b08c982535155d => generated 0 bytes in 283 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 05:37:20.295001 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a28d4243-10ef-4f1e-a8cc-4ffc80b871bd admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:37:20.305306 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-a28d4243-10ef-4f1e-a8cc-4ffc80b871bd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:37:20.374590 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 868/1718] 10.222.0.33 () {58 vars in 1009 bytes} [Mon Jul 25 05:37:20 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 263 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:37:20.375951 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca3507cd-1792-4cb2-bfcb-80f00e52d235 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:37:20.377695 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-ca3507cd-1792-4cb2-bfcb-80f00e52d235 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:37:20.467832 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a28d4243-10ef-4f1e-a8cc-4ffc80b871bd admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:37:20.481990 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5c3d43-a246-4a91-b594-02f65971482d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:37:20.482904 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5c3d43-a246-4a91-b594-02f65971482d admin admin] REQUEST_METHOD: `PUT` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:37:20.483194 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5c3d43-a246-4a91-b594-02f65971482d admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:37:20.483478 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5c3d43-a246-4a91-b594-02f65971482d admin admin] PATH_INFO: `/v3/projects/8b5056fa50f347ddb09bc097201ee4b9/users/59c9b2ed66a44f238b5908beec7638b8/roles/5074e74e35f54d4b9bb415c34999a82a` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:37:20.522801 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad5c3d43-a246-4a91-b594-02f65971482d admin admin] RBAC: Authorizing `identity:create_grant(project_id=8b5056fa50f347ddb09bc097201ee4b9, user_id=59c9b2ed66a44f238b5908beec7638b8, role_id=5074e74e35f54d4b9bb415c34999a82a)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:37:20.531895 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad5c3d43-a246-4a91-b594-02f65971482d admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:37:20.564507 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 851/1719] 10.222.0.33 () {60 vars in 1482 bytes} [Mon Jul 25 05:37:20 2022] PUT /identity/v3/projects/8b5056fa50f347ddb09bc097201ee4b9/users/59c9b2ed66a44f238b5908beec7638b8/roles/5074e74e35f54d4b9bb415c34999a82a => generated 0 bytes in 271 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 05:37:20.577831 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd036abe-b18d-42bd-b88b-1b00ebc40215 None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:37:20.580316 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd036abe-b18d-42bd-b88b-1b00ebc40215 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:37:20.580907 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca3507cd-1792-4cb2-bfcb-80f00e52d235 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:37:20.581421 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd036abe-b18d-42bd-b88b-1b00ebc40215 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:37:20.656131 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-cd036abe-b18d-42bd-b88b-1b00ebc40215 None None] MFA Rules not processed for user `59c9b2ed66a44f238b5908beec7638b8`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:37:20.657575 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-cd036abe-b18d-42bd-b88b-1b00ebc40215 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:37:20.667979 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:37:20.783216 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c16b5382-509f-488d-b92b-53d2de83daf6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:37:20.786356 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16b5382-509f-488d-b92b-53d2de83daf6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:37:20.786668 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16b5382-509f-488d-b92b-53d2de83daf6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:37:20.787578 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16b5382-509f-488d-b92b-53d2de83daf6 admin admin] PATH_INFO: `/v3/users/dd9bcd70fa9a4c27a2e1eb72218ca242` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:37:20.790989 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c16b5382-509f-488d-b92b-53d2de83daf6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=dd9bcd70fa9a4c27a2e1eb72218ca242)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:37:20.794470 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c16b5382-509f-488d-b92b-53d2de83daf6 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:37:20.857366 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 852/1720] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 25 05:37:20 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 281 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:37:20.858607 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd036abe-b18d-42bd-b88b-1b00ebc40215 None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:37:20.863853 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-cd036abe-b18d-42bd-b88b-1b00ebc40215 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:37:20.933444 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-c16b5382-509f-488d-b92b-53d2de83daf6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dd9bcd70fa9a4c27a2e1eb72218ca242'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:37:20.934569 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-c16b5382-509f-488d-b92b-53d2de83daf6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dd9bcd70fa9a4c27a2e1eb72218ca242'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:37:20.980459 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-c16b5382-509f-488d-b92b-53d2de83daf6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dd9bcd70fa9a4c27a2e1eb72218ca242'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:37:21.025292 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-c16b5382-509f-488d-b92b-53d2de83daf6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dd9bcd70fa9a4c27a2e1eb72218ca242'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:37:21.027035 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-c16b5382-509f-488d-b92b-53d2de83daf6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dd9bcd70fa9a4c27a2e1eb72218ca242'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:37:21.074033 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 869/1721] 10.222.0.33 () {58 vars in 1223 bytes} [Mon Jul 25 05:37:20 2022] DELETE /identity/v3/users/dd9bcd70fa9a4c27a2e1eb72218ca242 => generated 0 bytes in 699 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 05:37:21.077583 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c16b5382-509f-488d-b92b-53d2de83daf6 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:37:21.084835 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-c16b5382-509f-488d-b92b-53d2de83daf6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:37:21.115614 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=96,keystone:DELETE=2,keystone:INSERT=2 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:37:21.259587 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd036abe-b18d-42bd-b88b-1b00ebc40215 None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:37:21.308336 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c16b5382-509f-488d-b92b-53d2de83daf6 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:37:21.318682 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b7823e-9641-4211-9a0a-9b43b77fa537 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db57b478c25643dbaf936f90f6e17d0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92afba266cce4d1bad4134c351e74be3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:37:21.320999 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b7823e-9641-4211-9a0a-9b43b77fa537 service cinder] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:37:21.321291 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b7823e-9641-4211-9a0a-9b43b77fa537 service cinder] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:37:21.321570 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b7823e-9641-4211-9a0a-9b43b77fa537 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:37:21.332085 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-47b7823e-9641-4211-9a0a-9b43b77fa537 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:37:21.428713 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4923c337-630c-42ae-a414-aa70e837ca4a service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb135a8d5d7444cfbfa336115ba2ba44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92afba266cce4d1bad4134c351e74be3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:37:21.429460 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4923c337-630c-42ae-a414-aa70e837ca4a service swift] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:37:21.430598 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4923c337-630c-42ae-a414-aa70e837ca4a service swift] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:37:21.431469 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4923c337-630c-42ae-a414-aa70e837ca4a service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:37:21.438314 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-4923c337-630c-42ae-a414-aa70e837ca4a service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:37:21.468024 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:37:21.475999 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47b7823e-9641-4211-9a0a-9b43b77fa537 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:37:21.479238 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47b7823e-9641-4211-9a0a-9b43b77fa537 service cinder] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:37:21.503689 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 870/1722] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jul 25 05:37:21 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 429 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 05:37:21.567148 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4923c337-630c-42ae-a414-aa70e837ca4a service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:37:21.570991 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4923c337-630c-42ae-a414-aa70e837ca4a service swift] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:37:21.588679 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 853/1723] 10.222.0.33 () {60 vars in 1384 bytes} [Mon Jul 25 05:37:20 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 731 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 25 05:37:22.676891 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6cc91a-aa0b-45c8-a8f9-786a12bd6619 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:37:22.677950 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6cc91a-aa0b-45c8-a8f9-786a12bd6619 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:37:22.678473 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6cc91a-aa0b-45c8-a8f9-786a12bd6619 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:37:22.770909 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-6d6cc91a-aa0b-45c8-a8f9-786a12bd6619 None None] MFA Rules not processed for user `b2931200f99e43d79a50a9f6e02738e6`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:37:22.775316 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-6d6cc91a-aa0b-45c8-a8f9-786a12bd6619 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:37:22.958509 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 871/1724] 10.222.0.43 () {58 vars in 1015 bytes} [Mon Jul 25 05:37:22 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 286 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:37:22.970947 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4923c337-630c-42ae-a414-aa70e837ca4a service swift] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:37:22.973144 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-4923c337-630c-42ae-a414-aa70e837ca4a service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:37:22.990574 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d6cc91a-aa0b-45c8-a8f9-786a12bd6619 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:37:22.992293 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-6d6cc91a-aa0b-45c8-a8f9-786a12bd6619 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:37:23.180366 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4923c337-630c-42ae-a414-aa70e837ca4a service swift] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:37:23.193357 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eadae14-5dc8-481b-8d18-de36d315b359 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:37:23.194639 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eadae14-5dc8-481b-8d18-de36d315b359 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:37:23.196833 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eadae14-5dc8-481b-8d18-de36d315b359 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:37:23.197468 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eadae14-5dc8-481b-8d18-de36d315b359 admin admin] PATH_INFO: `/v3/users/00d1cb0b04034ce882be46b34eda6618` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:37:23.200906 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eadae14-5dc8-481b-8d18-de36d315b359 admin admin] RBAC: Authorizing `identity:delete_user(user_id=00d1cb0b04034ce882be46b34eda6618)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:37:23.206220 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eadae14-5dc8-481b-8d18-de36d315b359 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:37:23.246842 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d6cc91a-aa0b-45c8-a8f9-786a12bd6619 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:37:23.260244 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad0331c-6f6b-4e41-97c6-5044266774a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:37:23.267918 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad0331c-6f6b-4e41-97c6-5044266774a3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:37:23.271796 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad0331c-6f6b-4e41-97c6-5044266774a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:37:23.272243 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad0331c-6f6b-4e41-97c6-5044266774a3 admin admin] PATH_INFO: `/v3/projects/20403937bb684d8e9d29b6992bf4e9f6` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:37:23.274268 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bad0331c-6f6b-4e41-97c6-5044266774a3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=20403937bb684d8e9d29b6992bf4e9f6)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:37:23.277928 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bad0331c-6f6b-4e41-97c6-5044266774a3 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:37:23.283839 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:37:23.416379 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-7eadae14-5dc8-481b-8d18-de36d315b359 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '00d1cb0b04034ce882be46b34eda6618'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:37:23.502714 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-7eadae14-5dc8-481b-8d18-de36d315b359 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '00d1cb0b04034ce882be46b34eda6618'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:37:23.506729 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-7eadae14-5dc8-481b-8d18-de36d315b359 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '00d1cb0b04034ce882be46b34eda6618'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:37:23.537644 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-7eadae14-5dc8-481b-8d18-de36d315b359 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '00d1cb0b04034ce882be46b34eda6618'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:37:23.538647 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-7eadae14-5dc8-481b-8d18-de36d315b359 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '00d1cb0b04034ce882be46b34eda6618'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:37:23.568865 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-bad0331c-6f6b-4e41-97c6-5044266774a3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '20403937bb684d8e9d29b6992bf4e9f6', 'request_id': 'req-bad0331c-6f6b-4e41-97c6-5044266774a3'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:37:23.598371 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 854/1725] 10.222.0.33 () {58 vars in 1223 bytes} [Mon Jul 25 05:37:22 2022] DELETE /identity/v3/users/00d1cb0b04034ce882be46b34eda6618 => generated 0 bytes in 625 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 05:37:23.598972 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-bad0331c-6f6b-4e41-97c6-5044266774a3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '20403937bb684d8e9d29b6992bf4e9f6', 'request_id': 'req-bad0331c-6f6b-4e41-97c6-5044266774a3'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:37:23.599317 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eadae14-5dc8-481b-8d18-de36d315b359 admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:37:23.600764 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-7eadae14-5dc8-481b-8d18-de36d315b359 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:37:23.627744 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-bad0331c-6f6b-4e41-97c6-5044266774a3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '20403937bb684d8e9d29b6992bf4e9f6', 'request_id': 'req-bad0331c-6f6b-4e41-97c6-5044266774a3'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:37:23.665033 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-bad0331c-6f6b-4e41-97c6-5044266774a3 admin admin] The token cache is being invalidate because project 20403937bb684d8e9d29b6992bf4e9f6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80249) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 05:37:23.665561 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-bad0331c-6f6b-4e41-97c6-5044266774a3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:37:23.672390 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-bad0331c-6f6b-4e41-97c6-5044266774a3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:37:23.675532 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 872/1726] 10.222.0.33 () {58 vars in 1232 bytes} [Mon Jul 25 05:37:22 2022] DELETE /identity/v3/projects/20403937bb684d8e9d29b6992bf4e9f6 => generated 0 bytes in 685 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 05:37:23.676584 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad0331c-6f6b-4e41-97c6-5044266774a3 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:37:23.683332 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-bad0331c-6f6b-4e41-97c6-5044266774a3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:37:23.786795 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:37:23.790398 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eadae14-5dc8-481b-8d18-de36d315b359 admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:37:23.800288 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b546a44-1a11-4952-888c-c75870babf29 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b2931200f99e43d79a50a9f6e02738e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92afba266cce4d1bad4134c351e74be3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:37:23.801533 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b546a44-1a11-4952-888c-c75870babf29 service neutron] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:37:23.802978 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b546a44-1a11-4952-888c-c75870babf29 service neutron] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:37:23.803627 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b546a44-1a11-4952-888c-c75870babf29 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:37:23.806170 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-7b546a44-1a11-4952-888c-c75870babf29 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:37:23.826058 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b546a44-1a11-4952-888c-c75870babf29 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:37:23.831195 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b546a44-1a11-4952-888c-c75870babf29 service neutron] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:37:23.842974 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 855/1727] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jul 25 05:37:23 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 244 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 05:37:23.846079 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6870f00-e934-4a7b-8136-19c303c11ddc None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:37:23.846876 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6870f00-e934-4a7b-8136-19c303c11ddc None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:37:23.847761 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6870f00-e934-4a7b-8136-19c303c11ddc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:37:23.887918 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad0331c-6f6b-4e41-97c6-5044266774a3 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:37:23.889744 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-e6870f00-e934-4a7b-8136-19c303c11ddc None None] MFA Rules not processed for user `1eae540242d94c10b0ad33960a855f08`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:37:23.895569 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-e6870f00-e934-4a7b-8136-19c303c11ddc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:37:24.107175 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7645ea52-0e4f-40e6-9461-19583716739b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:37:24.110187 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7645ea52-0e4f-40e6-9461-19583716739b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:37:24.113229 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7645ea52-0e4f-40e6-9461-19583716739b admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:37:24.114562 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7645ea52-0e4f-40e6-9461-19583716739b admin admin] PATH_INFO: `/v3/projects/e6f640be419b4c05b9b614d5b12b1fda` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:37:24.123954 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7645ea52-0e4f-40e6-9461-19583716739b admin admin] RBAC: Authorizing `identity:delete_project(project_id=e6f640be419b4c05b9b614d5b12b1fda)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:37:24.132088 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7645ea52-0e4f-40e6-9461-19583716739b admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:37:24.139897 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 856/1728] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 25 05:37:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 296 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:37:24.142647 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37fb8f12-0d85-491a-927f-b1f5311e8c81 None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:37:24.143142 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37fb8f12-0d85-491a-927f-b1f5311e8c81 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:37:24.143605 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37fb8f12-0d85-491a-927f-b1f5311e8c81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:37:24.191841 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-37fb8f12-0d85-491a-927f-b1f5311e8c81 None None] MFA Rules not processed for user `1eae540242d94c10b0ad33960a855f08`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:37:24.197074 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-37fb8f12-0d85-491a-927f-b1f5311e8c81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:37:24.232045 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 857/1729] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 25 05:37:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:37:24.237975 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972d9a0e-a522-4b04-b94b-432a5f756814 None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:37:24.237975 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972d9a0e-a522-4b04-b94b-432a5f756814 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:37:24.237975 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972d9a0e-a522-4b04-b94b-432a5f756814 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:37:24.290102 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-972d9a0e-a522-4b04-b94b-432a5f756814 None None] MFA Rules not processed for user `1eae540242d94c10b0ad33960a855f08`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:37:24.294621 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-972d9a0e-a522-4b04-b94b-432a5f756814 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:37:24.325233 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 858/1730] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 25 05:37:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 05:37:24.329811 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-972d9a0e-a522-4b04-b94b-432a5f756814 None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:37:24.356427 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-972d9a0e-a522-4b04-b94b-432a5f756814 None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:37:24.384628 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-7645ea52-0e4f-40e6-9461-19583716739b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e6f640be419b4c05b9b614d5b12b1fda', 'request_id': 'req-7645ea52-0e4f-40e6-9461-19583716739b'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:37:24.429443 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-7645ea52-0e4f-40e6-9461-19583716739b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e6f640be419b4c05b9b614d5b12b1fda', 'request_id': 'req-7645ea52-0e4f-40e6-9461-19583716739b'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:37:24.430658 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:DELETE=8,keystone:SELECT=91,keystone:INSERT=1 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:37:24.446015 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-7645ea52-0e4f-40e6-9461-19583716739b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e6f640be419b4c05b9b614d5b12b1fda', 'request_id': 'req-7645ea52-0e4f-40e6-9461-19583716739b'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:37:24.483754 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-7645ea52-0e4f-40e6-9461-19583716739b admin admin] The token cache is being invalidate because project e6f640be419b4c05b9b614d5b12b1fda was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80249) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 05:37:24.484289 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-7645ea52-0e4f-40e6-9461-19583716739b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:37:24.486135 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-7645ea52-0e4f-40e6-9461-19583716739b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:37:24.489013 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 873/1731] 10.222.0.33 () {58 vars in 1232 bytes} [Mon Jul 25 05:37:23 2022] DELETE /identity/v3/projects/e6f640be419b4c05b9b614d5b12b1fda => generated 0 bytes in 813 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 05:37:24.498932 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7645ea52-0e4f-40e6-9461-19583716739b admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:37:24.500949 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-7645ea52-0e4f-40e6-9461-19583716739b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:37:24.531504 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c58ebd1c-c99c-4874-90ea-96e5fce91e70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:37:24.532609 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58ebd1c-c99c-4874-90ea-96e5fce91e70 admin admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:37:24.532952 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58ebd1c-c99c-4874-90ea-96e5fce91e70 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:37:24.533217 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58ebd1c-c99c-4874-90ea-96e5fce91e70 admin admin] PATH_INFO: `/v3/domains` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:37:24.534104 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c58ebd1c-c99c-4874-90ea-96e5fce91e70 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:37:24.534524 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c58ebd1c-c99c-4874-90ea-96e5fce91e70 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:37:24.540735 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c58ebd1c-c99c-4874-90ea-96e5fce91e70 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:37:24.600722 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 859/1732] 10.222.0.33 () {58 vars in 1152 bytes} [Mon Jul 25 05:37:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:37:24.600722 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c58ebd1c-c99c-4874-90ea-96e5fce91e70 admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:37:24.603039 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-c58ebd1c-c99c-4874-90ea-96e5fce91e70 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:37:24.770307 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7645ea52-0e4f-40e6-9461-19583716739b admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:37:24.772815 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c58ebd1c-c99c-4874-90ea-96e5fce91e70 admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:37:24.784690 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ddd883-e1a6-412c-91e4-7d02f376fa9c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b2931200f99e43d79a50a9f6e02738e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92afba266cce4d1bad4134c351e74be3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:37:24.786883 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ddd883-e1a6-412c-91e4-7d02f376fa9c service neutron] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:37:24.788032 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b4e6674-ab7c-4a53-9fca-1fbd339edd49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:37:24.788307 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ddd883-e1a6-412c-91e4-7d02f376fa9c service neutron] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:37:24.788934 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4e6674-ab7c-4a53-9fca-1fbd339edd49 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:37:24.789232 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4e6674-ab7c-4a53-9fca-1fbd339edd49 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:37:24.789521 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4e6674-ab7c-4a53-9fca-1fbd339edd49 admin admin] PATH_INFO: `/v3/users/f4686cad93e14779b611e747b458c0f2` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:37:24.790214 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ddd883-e1a6-412c-91e4-7d02f376fa9c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:37:24.794075 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b4e6674-ab7c-4a53-9fca-1fbd339edd49 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f4686cad93e14779b611e747b458c0f2)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:37:24.795059 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-47ddd883-e1a6-412c-91e4-7d02f376fa9c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:37:24.796225 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b4e6674-ab7c-4a53-9fca-1fbd339edd49 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:37:24.846957 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:37:24.848065 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47ddd883-e1a6-412c-91e4-7d02f376fa9c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:37:24.850587 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47ddd883-e1a6-412c-91e4-7d02f376fa9c service neutron] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:37:24.888225 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 860/1733] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jul 25 05:37:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 290 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 05:37:24.890424 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ddd883-e1a6-412c-91e4-7d02f376fa9c service neutron] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:37:24.893544 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-47ddd883-e1a6-412c-91e4-7d02f376fa9c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:37:24.922491 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-8b4e6674-ab7c-4a53-9fca-1fbd339edd49 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f4686cad93e14779b611e747b458c0f2'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:37:24.929792 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-8b4e6674-ab7c-4a53-9fca-1fbd339edd49 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f4686cad93e14779b611e747b458c0f2'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:37:24.930343 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ddd883-e1a6-412c-91e4-7d02f376fa9c service neutron] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:37:24.945275 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69bc2ced-f4ef-469b-b961-e13f884252e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:37:24.947143 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69bc2ced-f4ef-469b-b961-e13f884252e7 admin admin] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:37:24.947348 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69bc2ced-f4ef-469b-b961-e13f884252e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:37:24.947632 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69bc2ced-f4ef-469b-b961-e13f884252e7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:37:24.950019 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69bc2ced-f4ef-469b-b961-e13f884252e7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:37:24.953313 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69bc2ced-f4ef-469b-b961-e13f884252e7 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:37:24.954286 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-8b4e6674-ab7c-4a53-9fca-1fbd339edd49 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f4686cad93e14779b611e747b458c0f2'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:37:24.989911 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-8b4e6674-ab7c-4a53-9fca-1fbd339edd49 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f4686cad93e14779b611e747b458c0f2'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:37:24.991092 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-8b4e6674-ab7c-4a53-9fca-1fbd339edd49 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f4686cad93e14779b611e747b458c0f2'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:37:25.015536 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 874/1734] 10.222.0.33 () {58 vars in 1223 bytes} [Mon Jul 25 05:37:24 2022] DELETE /identity/v3/users/f4686cad93e14779b611e747b458c0f2 => generated 0 bytes in 514 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 05:37:25.041366 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b4e6674-ab7c-4a53-9fca-1fbd339edd49 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:37:25.044744 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-8b4e6674-ab7c-4a53-9fca-1fbd339edd49 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:37:25.049404 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 861/1735] 10.222.0.33 () {60 vars in 1152 bytes} [Mon Jul 25 05:37:24 2022] POST /identity/v3/projects => generated 367 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 05:37:25.104819 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69bc2ced-f4ef-469b-b961-e13f884252e7 admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:37:25.108203 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-69bc2ced-f4ef-469b-b961-e13f884252e7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:37:25.204311 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b4e6674-ab7c-4a53-9fca-1fbd339edd49 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:37:25.314422 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69bc2ced-f4ef-469b-b961-e13f884252e7 admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:37:25.323628 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-670bed2e-ce15-41cc-b9f4-9e56b95b9387 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:37:25.324911 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670bed2e-ce15-41cc-b9f4-9e56b95b9387 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:37:25.331052 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670bed2e-ce15-41cc-b9f4-9e56b95b9387 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:37:25.331526 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670bed2e-ce15-41cc-b9f4-9e56b95b9387 admin admin] PATH_INFO: `/v3/projects/aa82c2fa3dea499a8b5f1e7d5fcd8acd` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:37:25.333579 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-670bed2e-ce15-41cc-b9f4-9e56b95b9387 admin admin] RBAC: Authorizing `identity:delete_project(project_id=aa82c2fa3dea499a8b5f1e7d5fcd8acd)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:37:25.336441 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-670bed2e-ce15-41cc-b9f4-9e56b95b9387 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:37:25.347713 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b13af3c1-23ab-4524-be6f-10df6b2a2d7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:37:25.348808 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13af3c1-23ab-4524-be6f-10df6b2a2d7f admin admin] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:37:25.349516 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13af3c1-23ab-4524-be6f-10df6b2a2d7f admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:37:25.351204 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13af3c1-23ab-4524-be6f-10df6b2a2d7f admin admin] PATH_INFO: `/v3/users` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:37:25.352309 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b13af3c1-23ab-4524-be6f-10df6b2a2d7f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:37:25.357337 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b13af3c1-23ab-4524-be6f-10df6b2a2d7f admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:37:25.403227 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 875/1736] 10.222.0.33 () {60 vars in 1143 bytes} [Mon Jul 25 05:37:25 2022] POST /identity/v3/users => generated 284 bytes in 363 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 05:37:25.407453 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b13af3c1-23ab-4524-be6f-10df6b2a2d7f admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:37:25.434788 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b13af3c1-23ab-4524-be6f-10df6b2a2d7f admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:37:25.450357 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f42a8725-77e1-41eb-9ec5-309c732e537b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:37:25.451537 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42a8725-77e1-41eb-9ec5-309c732e537b admin admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:37:25.451976 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42a8725-77e1-41eb-9ec5-309c732e537b admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:37:25.452429 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42a8725-77e1-41eb-9ec5-309c732e537b admin admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:37:25.453668 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f42a8725-77e1-41eb-9ec5-309c732e537b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:37:25.456274 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f42a8725-77e1-41eb-9ec5-309c732e537b admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:37:25.469382 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=96,keystone:INSERT=1,keystone:DELETE=3 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:37:25.480074 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 876/1737] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jul 25 05:37:25 2022] GET /identity/v3/roles => generated 1420 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 25 05:37:25.484765 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f42a8725-77e1-41eb-9ec5-309c732e537b admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:37:25.485838 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-670bed2e-ce15-41cc-b9f4-9e56b95b9387 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'aa82c2fa3dea499a8b5f1e7d5fcd8acd', 'request_id': 'req-670bed2e-ce15-41cc-b9f4-9e56b95b9387'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:37:25.506120 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-670bed2e-ce15-41cc-b9f4-9e56b95b9387 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'aa82c2fa3dea499a8b5f1e7d5fcd8acd', 'request_id': 'req-670bed2e-ce15-41cc-b9f4-9e56b95b9387'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:37:25.518531 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f42a8725-77e1-41eb-9ec5-309c732e537b admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:37:25.527944 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-670bed2e-ce15-41cc-b9f4-9e56b95b9387 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'aa82c2fa3dea499a8b5f1e7d5fcd8acd', 'request_id': 'req-670bed2e-ce15-41cc-b9f4-9e56b95b9387'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:37:25.557062 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=87,keystone:DELETE=9,keystone:INSERT=4 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:37:25.569605 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-670bed2e-ce15-41cc-b9f4-9e56b95b9387 admin admin] The token cache is being invalidate because project aa82c2fa3dea499a8b5f1e7d5fcd8acd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80248) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 05:37:25.570191 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-670bed2e-ce15-41cc-b9f4-9e56b95b9387 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:37:25.574637 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-670bed2e-ce15-41cc-b9f4-9e56b95b9387 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:37:25.583342 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 862/1738] 10.222.0.33 () {58 vars in 1232 bytes} [Mon Jul 25 05:37:25 2022] DELETE /identity/v3/projects/aa82c2fa3dea499a8b5f1e7d5fcd8acd => generated 0 bytes in 480 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 25 05:37:25.594975 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545d91d6-015e-4dba-b0d5-7a8646adba08 None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:37:25.595386 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545d91d6-015e-4dba-b0d5-7a8646adba08 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:37:25.595807 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545d91d6-015e-4dba-b0d5-7a8646adba08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:37:25.667348 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-545d91d6-015e-4dba-b0d5-7a8646adba08 None None] MFA Rules not processed for user `1eae540242d94c10b0ad33960a855f08`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:37:25.670452 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-545d91d6-015e-4dba-b0d5-7a8646adba08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:37:25.688937 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a151463-b618-4d0c-bdf3-a56e9aea12b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:37:25.689741 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a151463-b618-4d0c-bdf3-a56e9aea12b8 admin admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:37:25.690189 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a151463-b618-4d0c-bdf3-a56e9aea12b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:37:25.690476 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a151463-b618-4d0c-bdf3-a56e9aea12b8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:37:25.691490 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a151463-b618-4d0c-bdf3-a56e9aea12b8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:37:25.693068 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a151463-b618-4d0c-bdf3-a56e9aea12b8 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:37:25.719892 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 877/1739] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jul 25 05:37:25 2022] GET /identity/v3/roles => generated 1420 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 05:37:25.721087 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a151463-b618-4d0c-bdf3-a56e9aea12b8 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:37:25.730072 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-7a151463-b618-4d0c-bdf3-a56e9aea12b8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:37:25.773489 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a151463-b618-4d0c-bdf3-a56e9aea12b8 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:37:25.795593 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d766f54-321d-41c9-8a69-3b0faf6db985 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:37:25.796510 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d766f54-321d-41c9-8a69-3b0faf6db985 admin admin] REQUEST_METHOD: `PUT` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:37:25.796811 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d766f54-321d-41c9-8a69-3b0faf6db985 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:37:25.802366 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d766f54-321d-41c9-8a69-3b0faf6db985 admin admin] PATH_INFO: `/v3/projects/65bae6dda085445d8ab9e7103b920e15/users/a8ed129b857a4aff954e16c1d2ec29ae/roles/5074e74e35f54d4b9bb415c34999a82a` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:37:25.858191 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 863/1740] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 25 05:37:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 269 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:37:25.866501 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e01000-7555-40bc-a96c-04a7e16a5f1b None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:37:25.866816 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e01000-7555-40bc-a96c-04a7e16a5f1b None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:37:25.867078 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e01000-7555-40bc-a96c-04a7e16a5f1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:37:25.878753 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d766f54-321d-41c9-8a69-3b0faf6db985 admin admin] RBAC: Authorizing `identity:create_grant(project_id=65bae6dda085445d8ab9e7103b920e15, user_id=a8ed129b857a4aff954e16c1d2ec29ae, role_id=5074e74e35f54d4b9bb415c34999a82a)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:37:25.894547 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d766f54-321d-41c9-8a69-3b0faf6db985 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:37:25.913612 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-53e01000-7555-40bc-a96c-04a7e16a5f1b None None] MFA Rules not processed for user `1eae540242d94c10b0ad33960a855f08`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:37:25.915150 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-53e01000-7555-40bc-a96c-04a7e16a5f1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:37:25.928961 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 878/1741] 10.222.0.33 () {60 vars in 1482 bytes} [Mon Jul 25 05:37:25 2022] PUT /identity/v3/projects/65bae6dda085445d8ab9e7103b920e15/users/a8ed129b857a4aff954e16c1d2ec29ae/roles/5074e74e35f54d4b9bb415c34999a82a => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 05:37:25.944932 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5cb804-f53c-4670-8f23-db378667781f None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:37:25.945457 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5cb804-f53c-4670-8f23-db378667781f None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:37:25.946424 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5cb804-f53c-4670-8f23-db378667781f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:37:26.058187 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-2e5cb804-f53c-4670-8f23-db378667781f None None] MFA Rules not processed for user `a8ed129b857a4aff954e16c1d2ec29ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:37:26.059673 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-2e5cb804-f53c-4670-8f23-db378667781f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:37:26.178717 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 864/1742] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 25 05:37:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 312 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 05:37:26.187151 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e01000-7555-40bc-a96c-04a7e16a5f1b None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:37:26.203355 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e01000-7555-40bc-a96c-04a7e16a5f1b None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:37:26.225796 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-059bdef0-452d-4243-a58e-ba363b39bb0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:37:26.226278 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059bdef0-452d-4243-a58e-ba363b39bb0c admin admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:37:26.226570 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059bdef0-452d-4243-a58e-ba363b39bb0c admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:37:26.227230 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059bdef0-452d-4243-a58e-ba363b39bb0c admin admin] PATH_INFO: `/v3/domains` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:37:26.228393 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-059bdef0-452d-4243-a58e-ba363b39bb0c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:37:26.228922 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-059bdef0-452d-4243-a58e-ba363b39bb0c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:37:26.230640 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-059bdef0-452d-4243-a58e-ba363b39bb0c admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:37:26.302410 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 865/1743] 10.222.0.33 () {58 vars in 1152 bytes} [Mon Jul 25 05:37:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 123 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:37:26.303502 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-059bdef0-452d-4243-a58e-ba363b39bb0c admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:37:26.332321 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 879/1744] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 25 05:37:25 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 390 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 05:37:26.337584 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e5cb804-f53c-4670-8f23-db378667781f None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:37:26.343345 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-059bdef0-452d-4243-a58e-ba363b39bb0c admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:37:26.372608 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4863453-a7f9-47d3-9d61-5d71767da488 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:37:26.373341 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4863453-a7f9-47d3-9d61-5d71767da488 admin admin] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:37:26.373629 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4863453-a7f9-47d3-9d61-5d71767da488 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:37:26.373907 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4863453-a7f9-47d3-9d61-5d71767da488 admin admin] PATH_INFO: `/v3/projects` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:37:26.374626 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4863453-a7f9-47d3-9d61-5d71767da488 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:37:26.376574 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4863453-a7f9-47d3-9d61-5d71767da488 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:37:26.379958 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e5cb804-f53c-4670-8f23-db378667781f None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:37:26.399344 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee1f6252-ff77-4afe-b35e-748a35a6b844 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:37:26.401094 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1f6252-ff77-4afe-b35e-748a35a6b844 admin admin] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:37:26.401991 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1f6252-ff77-4afe-b35e-748a35a6b844 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:37:26.402278 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1f6252-ff77-4afe-b35e-748a35a6b844 admin admin] PATH_INFO: `/v3/projects` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:37:26.403012 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee1f6252-ff77-4afe-b35e-748a35a6b844 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:37:26.417734 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee1f6252-ff77-4afe-b35e-748a35a6b844 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:37:26.498279 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 880/1745] 10.222.0.33 () {60 vars in 1152 bytes} [Mon Jul 25 05:37:26 2022] POST /identity/v3/projects => generated 367 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 05:37:26.501964 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 866/1746] 10.222.0.33 () {60 vars in 1152 bytes} [Mon Jul 25 05:37:26 2022] POST /identity/v3/projects => generated 365 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 05:37:26.502282 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee1f6252-ff77-4afe-b35e-748a35a6b844 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:37:26.503782 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4863453-a7f9-47d3-9d61-5d71767da488 admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:37:26.613149 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=94,keystone:DELETE=4,keystone:INSERT=2 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:37:26.688040 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4863453-a7f9-47d3-9d61-5d71767da488 admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:37:26.706827 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c59d47a1-b528-4611-b7a9-b3685452907d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:37:26.707690 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59d47a1-b528-4611-b7a9-b3685452907d admin admin] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:37:26.708141 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59d47a1-b528-4611-b7a9-b3685452907d admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:37:26.709984 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59d47a1-b528-4611-b7a9-b3685452907d admin admin] PATH_INFO: `/v3/users` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:37:26.711059 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c59d47a1-b528-4611-b7a9-b3685452907d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:37:26.720550 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c59d47a1-b528-4611-b7a9-b3685452907d admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:37:26.764109 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:37:26.774754 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 867/1747] 10.222.0.33 () {60 vars in 1143 bytes} [Mon Jul 25 05:37:26 2022] POST /identity/v3/users => generated 290 bytes in 268 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 05:37:26.775160 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee1f6252-ff77-4afe-b35e-748a35a6b844 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:37:26.780208 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c59d47a1-b528-4611-b7a9-b3685452907d admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:37:26.790128 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce83389-6ce0-407e-8372-03916cee55c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:37:26.791074 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce83389-6ce0-407e-8372-03916cee55c7 admin admin] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:37:26.791419 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce83389-6ce0-407e-8372-03916cee55c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:37:26.793454 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce83389-6ce0-407e-8372-03916cee55c7 admin admin] PATH_INFO: `/v3/users` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:37:26.794617 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ce83389-6ce0-407e-8372-03916cee55c7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:37:26.796753 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ce83389-6ce0-407e-8372-03916cee55c7 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:37:26.816300 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c59d47a1-b528-4611-b7a9-b3685452907d admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:37:26.831702 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d580c2-ec59-4973-ac3b-e0cb66626f99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:37:26.832511 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d580c2-ec59-4973-ac3b-e0cb66626f99 admin admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:37:26.832821 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d580c2-ec59-4973-ac3b-e0cb66626f99 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:37:26.833108 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d580c2-ec59-4973-ac3b-e0cb66626f99 admin admin] PATH_INFO: `/v3/roles` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:37:26.833880 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8d580c2-ec59-4973-ac3b-e0cb66626f99 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:37:26.842427 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8d580c2-ec59-4973-ac3b-e0cb66626f99 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:37:26.867153 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 868/1748] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jul 25 05:37:26 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 05:37:26.868253 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d580c2-ec59-4973-ac3b-e0cb66626f99 admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:37:26.888360 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d580c2-ec59-4973-ac3b-e0cb66626f99 admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:37:26.901638 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 881/1749] 10.222.0.33 () {60 vars in 1143 bytes} [Mon Jul 25 05:37:26 2022] POST /identity/v3/users => generated 283 bytes in 397 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 05:37:26.902863 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce83389-6ce0-407e-8372-03916cee55c7 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:37:26.909289 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-380b8666-0214-4bbf-8a36-e1f03a653974 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:37:26.910460 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380b8666-0214-4bbf-8a36-e1f03a653974 admin admin] REQUEST_METHOD: `PUT` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:37:26.910958 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380b8666-0214-4bbf-8a36-e1f03a653974 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:37:26.912140 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380b8666-0214-4bbf-8a36-e1f03a653974 admin admin] PATH_INFO: `/v3/projects/6d5fd404b69e41ac8af538fd4b4d914f/users/8b6883c886294f6f9fe5dc09b2784ff2/roles/39e5c666285a42fe8fc2c3afca6260d1` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:37:26.932871 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce83389-6ce0-407e-8372-03916cee55c7 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:37:26.957918 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72634f02-b7fe-42da-97af-9725cbe8ef1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:37:26.958825 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72634f02-b7fe-42da-97af-9725cbe8ef1b admin admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:37:26.959194 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72634f02-b7fe-42da-97af-9725cbe8ef1b admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:37:26.959548 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72634f02-b7fe-42da-97af-9725cbe8ef1b admin admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:37:26.960392 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72634f02-b7fe-42da-97af-9725cbe8ef1b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:37:26.961548 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-380b8666-0214-4bbf-8a36-e1f03a653974 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6d5fd404b69e41ac8af538fd4b4d914f, user_id=8b6883c886294f6f9fe5dc09b2784ff2, role_id=39e5c666285a42fe8fc2c3afca6260d1)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:37:26.963482 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72634f02-b7fe-42da-97af-9725cbe8ef1b admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:37:26.968858 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-380b8666-0214-4bbf-8a36-e1f03a653974 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:37:26.997687 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 869/1750] 10.222.0.33 () {60 vars in 1482 bytes} [Mon Jul 25 05:37:26 2022] PUT /identity/v3/projects/6d5fd404b69e41ac8af538fd4b4d914f/users/8b6883c886294f6f9fe5dc09b2784ff2/roles/39e5c666285a42fe8fc2c3afca6260d1 => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 05:37:27.001512 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-380b8666-0214-4bbf-8a36-e1f03a653974 admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:37:27.025023 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 882/1751] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jul 25 05:37:26 2022] GET /identity/v3/roles => generated 1420 bytes in 119 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 05:37:27.026808 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72634f02-b7fe-42da-97af-9725cbe8ef1b admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:37:27.060890 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72634f02-b7fe-42da-97af-9725cbe8ef1b admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:37:27.154752 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-380b8666-0214-4bbf-8a36-e1f03a653974 admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:37:27.169576 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a3b2a6-6a52-4604-8b58-113e9a6de300 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:37:27.170731 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a3b2a6-6a52-4604-8b58-113e9a6de300 admin admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:37:27.171097 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a3b2a6-6a52-4604-8b58-113e9a6de300 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:37:27.172296 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a3b2a6-6a52-4604-8b58-113e9a6de300 admin admin] PATH_INFO: `/v3/roles` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:37:27.176005 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8a3b2a6-6a52-4604-8b58-113e9a6de300 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:37:27.180089 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8a3b2a6-6a52-4604-8b58-113e9a6de300 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:37:27.219413 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 870/1752] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jul 25 05:37:27 2022] GET /identity/v3/roles => generated 1420 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 05:37:27.219413 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a3b2a6-6a52-4604-8b58-113e9a6de300 admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:37:27.249306 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a3b2a6-6a52-4604-8b58-113e9a6de300 admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:37:27.249526 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a3cc794-9981-4a78-b3ef-bb70d728357f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:37:27.250192 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3cc794-9981-4a78-b3ef-bb70d728357f admin admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:37:27.250551 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3cc794-9981-4a78-b3ef-bb70d728357f admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:37:27.250765 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3cc794-9981-4a78-b3ef-bb70d728357f admin admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:37:27.251523 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a3cc794-9981-4a78-b3ef-bb70d728357f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:37:27.253185 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a3cc794-9981-4a78-b3ef-bb70d728357f admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:37:27.269804 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f16e8f-ee76-4d67-b53b-35f12774d9d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:37:27.270466 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f16e8f-ee76-4d67-b53b-35f12774d9d7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:37:27.270750 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f16e8f-ee76-4d67-b53b-35f12774d9d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:37:27.271030 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f16e8f-ee76-4d67-b53b-35f12774d9d7 admin admin] PATH_INFO: `/v3/projects/6d5fd404b69e41ac8af538fd4b4d914f/users/8b6883c886294f6f9fe5dc09b2784ff2/roles/39e5c666285a42fe8fc2c3afca6260d1` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:37:27.273342 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65f16e8f-ee76-4d67-b53b-35f12774d9d7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6d5fd404b69e41ac8af538fd4b4d914f, user_id=8b6883c886294f6f9fe5dc09b2784ff2, role_id=39e5c666285a42fe8fc2c3afca6260d1)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:37:27.277983 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65f16e8f-ee76-4d67-b53b-35f12774d9d7 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:37:27.279871 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 883/1753] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jul 25 05:37:27 2022] GET /identity/v3/roles => generated 1420 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 05:37:27.289955 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a3cc794-9981-4a78-b3ef-bb70d728357f admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:37:27.321057 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 871/1754] 10.222.0.33 () {60 vars in 1482 bytes} [Mon Jul 25 05:37:27 2022] PUT /identity/v3/projects/6d5fd404b69e41ac8af538fd4b4d914f/users/8b6883c886294f6f9fe5dc09b2784ff2/roles/39e5c666285a42fe8fc2c3afca6260d1 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 05:37:27.324452 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a3cc794-9981-4a78-b3ef-bb70d728357f admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:37:27.335055 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8d3d18-6d44-4b64-8980-f9582f761794 None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:37:27.336354 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8d3d18-6d44-4b64-8980-f9582f761794 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:37:27.336642 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8d3d18-6d44-4b64-8980-f9582f761794 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:37:27.403611 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-ef8d3d18-6d44-4b64-8980-f9582f761794 None None] MFA Rules not processed for user `8b6883c886294f6f9fe5dc09b2784ff2`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:37:27.405797 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-ef8d3d18-6d44-4b64-8980-f9582f761794 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:37:27.493148 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d34ab6-58e4-48d1-941b-111310893de6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:37:27.496413 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d34ab6-58e4-48d1-941b-111310893de6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:37:27.496995 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d34ab6-58e4-48d1-941b-111310893de6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:37:27.497287 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d34ab6-58e4-48d1-941b-111310893de6 admin admin] PATH_INFO: `/v3/projects/c82715a2535c4d369676883b34e936e7/users/b0fba4dd135043998a2e3a4e8837d10d/roles/5074e74e35f54d4b9bb415c34999a82a` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:37:27.607885 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79d34ab6-58e4-48d1-941b-111310893de6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c82715a2535c4d369676883b34e936e7, user_id=b0fba4dd135043998a2e3a4e8837d10d, role_id=5074e74e35f54d4b9bb415c34999a82a)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:37:27.610794 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79d34ab6-58e4-48d1-941b-111310893de6 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:37:27.634110 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:37:27.646283 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 884/1755] 10.222.0.33 () {60 vars in 1482 bytes} [Mon Jul 25 05:37:27 2022] PUT /identity/v3/projects/c82715a2535c4d369676883b34e936e7/users/b0fba4dd135043998a2e3a4e8837d10d/roles/5074e74e35f54d4b9bb415c34999a82a => generated 0 bytes in 357 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 05:37:27.664194 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebdf387c-18fd-4e86-9fa2-d129bc6670a1 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:37:27.664491 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebdf387c-18fd-4e86-9fa2-d129bc6670a1 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:37:27.664838 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebdf387c-18fd-4e86-9fa2-d129bc6670a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:37:27.665876 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 872/1756] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 25 05:37:27 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 332 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 05:37:27.703672 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef8d3d18-6d44-4b64-8980-f9582f761794 None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:37:27.706514 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-ef8d3d18-6d44-4b64-8980-f9582f761794 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:37:27.738645 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-ebdf387c-18fd-4e86-9fa2-d129bc6670a1 None None] MFA Rules not processed for user `b0fba4dd135043998a2e3a4e8837d10d`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:37:27.740155 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-ebdf387c-18fd-4e86-9fa2-d129bc6670a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:37:27.844505 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:37:27.892545 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 885/1757] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 25 05:37:27 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 230 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:37:27.899185 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebdf387c-18fd-4e86-9fa2-d129bc6670a1 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:37:27.985681 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef8d3d18-6d44-4b64-8980-f9582f761794 None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:37:28.162738 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebdf387c-18fd-4e86-9fa2-d129bc6670a1 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:37:28.184993 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c69e9a1-d675-4943-9ca2-1b7d9bae216e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:37:28.185791 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c69e9a1-d675-4943-9ca2-1b7d9bae216e admin admin] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:37:28.186095 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c69e9a1-d675-4943-9ca2-1b7d9bae216e admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:37:28.186354 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c69e9a1-d675-4943-9ca2-1b7d9bae216e admin admin] PATH_INFO: `/v3/projects` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:37:28.187074 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c69e9a1-d675-4943-9ca2-1b7d9bae216e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:37:28.190191 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8469e5e-992e-4caf-bc93-ea497b42548d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db57b478c25643dbaf936f90f6e17d0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92afba266cce4d1bad4134c351e74be3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:37:28.191305 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8469e5e-992e-4caf-bc93-ea497b42548d service cinder] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:37:28.191921 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8469e5e-992e-4caf-bc93-ea497b42548d service cinder] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:37:28.192440 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8469e5e-992e-4caf-bc93-ea497b42548d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:37:28.194280 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c69e9a1-d675-4943-9ca2-1b7d9bae216e admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:37:28.275561 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 886/1758] 10.222.0.33 () {60 vars in 1152 bytes} [Mon Jul 25 05:37:27 2022] POST /identity/v3/projects => generated 365 bytes in 379 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 05:37:28.280614 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c69e9a1-d675-4943-9ca2-1b7d9bae216e admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:37:28.415004 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8469e5e-992e-4caf-bc93-ea497b42548d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:37:28.416408 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8469e5e-992e-4caf-bc93-ea497b42548d service cinder] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:37:28.447738 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 873/1759] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jul 25 05:37:27 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 745 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 25 05:37:28.450857 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c69e9a1-d675-4943-9ca2-1b7d9bae216e admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:37:28.469252 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a50db13-c936-4417-ab89-cb0f0e05019e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:37:28.470805 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a50db13-c936-4417-ab89-cb0f0e05019e admin admin] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:37:28.471698 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a50db13-c936-4417-ab89-cb0f0e05019e admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:37:28.472099 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a50db13-c936-4417-ab89-cb0f0e05019e admin admin] PATH_INFO: `/v3/users` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:37:28.475083 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a50db13-c936-4417-ab89-cb0f0e05019e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:37:28.477253 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a50db13-c936-4417-ab89-cb0f0e05019e admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:37:28.562433 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 887/1760] 10.222.0.33 () {60 vars in 1143 bytes} [Mon Jul 25 05:37:28 2022] POST /identity/v3/users => generated 289 bytes in 282 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 05:37:28.566742 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8469e5e-992e-4caf-bc93-ea497b42548d service cinder] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:37:28.596102 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8469e5e-992e-4caf-bc93-ea497b42548d service cinder] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:37:28.607809 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa0c2aa8-db52-47b0-868f-375746749f75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:37:28.608502 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0c2aa8-db52-47b0-868f-375746749f75 admin admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:37:28.608821 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0c2aa8-db52-47b0-868f-375746749f75 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:37:28.611756 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0c2aa8-db52-47b0-868f-375746749f75 admin admin] PATH_INFO: `/v3/roles` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:37:28.613343 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa0c2aa8-db52-47b0-868f-375746749f75 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:37:28.614978 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa0c2aa8-db52-47b0-868f-375746749f75 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:37:28.649556 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 874/1761] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jul 25 05:37:28 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 05:37:28.654064 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a50db13-c936-4417-ab89-cb0f0e05019e admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:37:28.670089 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a50db13-c936-4417-ab89-cb0f0e05019e admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:37:28.684932 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb7bd1c-c5db-402c-8495-7eb0b30846b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:37:28.685998 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb7bd1c-c5db-402c-8495-7eb0b30846b1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:37:28.686166 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb7bd1c-c5db-402c-8495-7eb0b30846b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:37:28.686441 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb7bd1c-c5db-402c-8495-7eb0b30846b1 admin admin] PATH_INFO: `/v3/projects/9555a3211d264bc5beeefbe68cccc4e1/users/e698e554d5c8486e9805f49c9671d2aa/roles/39e5c666285a42fe8fc2c3afca6260d1` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:37:28.725121 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa0c2aa8-db52-47b0-868f-375746749f75 admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:37:28.727872 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-aa0c2aa8-db52-47b0-868f-375746749f75 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:37:28.759321 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa0c2aa8-db52-47b0-868f-375746749f75 admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:37:28.788058 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbb7bd1c-c5db-402c-8495-7eb0b30846b1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9555a3211d264bc5beeefbe68cccc4e1, user_id=e698e554d5c8486e9805f49c9671d2aa, role_id=39e5c666285a42fe8fc2c3afca6260d1)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:37:28.794855 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8758ea35-9d68-4790-8f81-bafd1ff0d4c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:37:28.796724 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8758ea35-9d68-4790-8f81-bafd1ff0d4c7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:37:28.797166 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8758ea35-9d68-4790-8f81-bafd1ff0d4c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:37:28.797563 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8758ea35-9d68-4790-8f81-bafd1ff0d4c7 admin admin] PATH_INFO: `/v3/projects/2b382be51c0a41d38d8b345038ed3daf` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:37:28.800066 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8758ea35-9d68-4790-8f81-bafd1ff0d4c7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2b382be51c0a41d38d8b345038ed3daf)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:37:28.802304 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8758ea35-9d68-4790-8f81-bafd1ff0d4c7 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:37:28.821243 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbb7bd1c-c5db-402c-8495-7eb0b30846b1 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:37:28.870061 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 888/1762] 10.222.0.33 () {60 vars in 1482 bytes} [Mon Jul 25 05:37:28 2022] PUT /identity/v3/projects/9555a3211d264bc5beeefbe68cccc4e1/users/e698e554d5c8486e9805f49c9671d2aa/roles/39e5c666285a42fe8fc2c3afca6260d1 => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 05:37:28.876300 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb7bd1c-c5db-402c-8495-7eb0b30846b1 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:37:28.915968 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-8758ea35-9d68-4790-8f81-bafd1ff0d4c7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2b382be51c0a41d38d8b345038ed3daf', 'request_id': 'req-8758ea35-9d68-4790-8f81-bafd1ff0d4c7'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:37:28.920026 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:37:28.983811 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-8758ea35-9d68-4790-8f81-bafd1ff0d4c7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2b382be51c0a41d38d8b345038ed3daf', 'request_id': 'req-8758ea35-9d68-4790-8f81-bafd1ff0d4c7'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:37:29.009617 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-8758ea35-9d68-4790-8f81-bafd1ff0d4c7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2b382be51c0a41d38d8b345038ed3daf', 'request_id': 'req-8758ea35-9d68-4790-8f81-bafd1ff0d4c7'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:37:29.034631 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:37:29.065860 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb7bd1c-c5db-402c-8495-7eb0b30846b1 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:37:29.068439 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-8758ea35-9d68-4790-8f81-bafd1ff0d4c7 admin admin] The token cache is being invalidate because project 2b382be51c0a41d38d8b345038ed3daf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80248) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 05:37:29.071881 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-8758ea35-9d68-4790-8f81-bafd1ff0d4c7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:37:29.077681 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-8758ea35-9d68-4790-8f81-bafd1ff0d4c7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:37:29.080945 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 875/1763] 10.222.0.33 () {58 vars in 1232 bytes} [Mon Jul 25 05:37:28 2022] DELETE /identity/v3/projects/2b382be51c0a41d38d8b345038ed3daf => generated 0 bytes in 357 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 05:37:29.091756 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3efd1212-6d99-4279-984f-9ee87839f32a None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:37:29.096636 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3efd1212-6d99-4279-984f-9ee87839f32a None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:37:29.099713 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3efd1212-6d99-4279-984f-9ee87839f32a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:37:29.166248 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-3efd1212-6d99-4279-984f-9ee87839f32a None None] MFA Rules not processed for user `1eae540242d94c10b0ad33960a855f08`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:37:29.175419 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-3efd1212-6d99-4279-984f-9ee87839f32a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:37:29.260283 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d332999b-b4aa-4ed5-a2fa-f2b4fa147a93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:37:29.261278 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d332999b-b4aa-4ed5-a2fa-f2b4fa147a93 admin admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:37:29.261677 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d332999b-b4aa-4ed5-a2fa-f2b4fa147a93 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:37:29.262062 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d332999b-b4aa-4ed5-a2fa-f2b4fa147a93 admin admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:37:29.262930 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d332999b-b4aa-4ed5-a2fa-f2b4fa147a93 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:37:29.264672 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d332999b-b4aa-4ed5-a2fa-f2b4fa147a93 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:37:29.300200 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 889/1764] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jul 25 05:37:28 2022] GET /identity/v3/roles => generated 1420 bytes in 420 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 05:37:29.303601 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d332999b-b4aa-4ed5-a2fa-f2b4fa147a93 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:37:29.307865 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-d332999b-b4aa-4ed5-a2fa-f2b4fa147a93 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:37:29.331538 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d332999b-b4aa-4ed5-a2fa-f2b4fa147a93 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:37:29.341923 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d346e0f-032f-41b5-9fce-cd9943bc861a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:37:29.344336 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d346e0f-032f-41b5-9fce-cd9943bc861a admin admin] REQUEST_METHOD: `PUT` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:37:29.344764 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d346e0f-032f-41b5-9fce-cd9943bc861a admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:37:29.345122 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d346e0f-032f-41b5-9fce-cd9943bc861a admin admin] PATH_INFO: `/v3/projects/9555a3211d264bc5beeefbe68cccc4e1/users/e698e554d5c8486e9805f49c9671d2aa/roles/39e5c666285a42fe8fc2c3afca6260d1` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:37:29.348712 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d346e0f-032f-41b5-9fce-cd9943bc861a admin admin] RBAC: Authorizing `identity:create_grant(project_id=9555a3211d264bc5beeefbe68cccc4e1, user_id=e698e554d5c8486e9805f49c9671d2aa, role_id=39e5c666285a42fe8fc2c3afca6260d1)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:37:29.353623 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d346e0f-032f-41b5-9fce-cd9943bc861a admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:37:29.379865 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 890/1765] 10.222.0.33 () {60 vars in 1482 bytes} [Mon Jul 25 05:37:29 2022] PUT /identity/v3/projects/9555a3211d264bc5beeefbe68cccc4e1/users/e698e554d5c8486e9805f49c9671d2aa/roles/39e5c666285a42fe8fc2c3afca6260d1 => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 05:37:29.380068 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 876/1766] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 25 05:37:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 287 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:37:29.383144 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405a992f-c930-4fe9-8e30-87ee15c7976b None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:37:29.384933 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405a992f-c930-4fe9-8e30-87ee15c7976b None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:37:29.385166 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405a992f-c930-4fe9-8e30-87ee15c7976b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:37:29.389854 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59a7e9b-5865-4110-9aba-8ec537f9b57b None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:37:29.390750 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59a7e9b-5865-4110-9aba-8ec537f9b57b None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:37:29.393134 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59a7e9b-5865-4110-9aba-8ec537f9b57b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:37:29.427329 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-d59a7e9b-5865-4110-9aba-8ec537f9b57b None None] MFA Rules not processed for user `1eae540242d94c10b0ad33960a855f08`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:37:29.429467 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-d59a7e9b-5865-4110-9aba-8ec537f9b57b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:37:29.535974 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-405a992f-c930-4fe9-8e30-87ee15c7976b None None] MFA Rules not processed for user `e698e554d5c8486e9805f49c9671d2aa`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:37:29.539487 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-405a992f-c930-4fe9-8e30-87ee15c7976b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:37:29.699712 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 877/1767] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 25 05:37:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 312 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:37:29.699712 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b669e10-e2e4-4169-9b36-de8acbcf37d3 None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:37:29.699712 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b669e10-e2e4-4169-9b36-de8acbcf37d3 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:37:29.699712 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b669e10-e2e4-4169-9b36-de8acbcf37d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:37:29.788676 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 891/1768] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 25 05:37:29 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 406 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:37:29.794350 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9eab496-da39-4a99-a14d-1b2a1d018932 None None] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:37:29.795936 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9eab496-da39-4a99-a14d-1b2a1d018932 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:37:29.800325 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9eab496-da39-4a99-a14d-1b2a1d018932 None None] PATH_INFO: `/` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:37:29.808216 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 892/1769] 10.222.0.33 () {54 vars in 921 bytes} [Mon Jul 25 05:37:29 2022] GET /identity => generated 270 bytes in 19 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:37:29.811343 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9eab496-da39-4a99-a14d-1b2a1d018932 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:37:29.821370 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-b9eab496-da39-4a99-a14d-1b2a1d018932 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:37:29.825012 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-5b669e10-e2e4-4169-9b36-de8acbcf37d3 None None] MFA Rules not processed for user `69f3276517de4f889dddd3180436ede9`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:37:29.826464 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-5b669e10-e2e4-4169-9b36-de8acbcf37d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:37:29.885379 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9eab496-da39-4a99-a14d-1b2a1d018932 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:37:29.921047 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb84e953-49b4-454f-92de-60c7959ba6cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:37:29.922689 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb84e953-49b4-454f-92de-60c7959ba6cf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:37:29.923085 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb84e953-49b4-454f-92de-60c7959ba6cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:37:29.923473 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb84e953-49b4-454f-92de-60c7959ba6cf admin admin] PATH_INFO: `/v3/projects/5c72716a74ac49f58d962efcba3e9962` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:37:29.926563 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb84e953-49b4-454f-92de-60c7959ba6cf admin admin] RBAC: Authorizing `identity:delete_project(project_id=5c72716a74ac49f58d962efcba3e9962)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:37:29.928970 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb84e953-49b4-454f-92de-60c7959ba6cf admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:37:30.005120 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 878/1770] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 25 05:37:29 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 308 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:37:30.006057 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b669e10-e2e4-4169-9b36-de8acbcf37d3 None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:37:30.058869 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b669e10-e2e4-4169-9b36-de8acbcf37d3 None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:37:30.065293 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=96,keystone:INSERT=1,keystone:DELETE=3 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:37:30.071699 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:DELETE=3,keystone:SELECT=96,keystone:INSERT=1 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:37:30.073492 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-cb84e953-49b4-454f-92de-60c7959ba6cf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5c72716a74ac49f58d962efcba3e9962', 'request_id': 'req-cb84e953-49b4-454f-92de-60c7959ba6cf'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:37:30.093106 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-cb84e953-49b4-454f-92de-60c7959ba6cf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5c72716a74ac49f58d962efcba3e9962', 'request_id': 'req-cb84e953-49b4-454f-92de-60c7959ba6cf'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:37:30.110024 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-cb84e953-49b4-454f-92de-60c7959ba6cf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5c72716a74ac49f58d962efcba3e9962', 'request_id': 'req-cb84e953-49b4-454f-92de-60c7959ba6cf'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:37:30.153044 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-cb84e953-49b4-454f-92de-60c7959ba6cf admin admin] The token cache is being invalidate because project 5c72716a74ac49f58d962efcba3e9962 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80249) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 05:37:30.153482 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-cb84e953-49b4-454f-92de-60c7959ba6cf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:37:30.155359 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-cb84e953-49b4-454f-92de-60c7959ba6cf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:37:30.158379 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 893/1771] 10.222.0.33 () {58 vars in 1232 bytes} [Mon Jul 25 05:37:29 2022] DELETE /identity/v3/projects/5c72716a74ac49f58d962efcba3e9962 => generated 0 bytes in 349 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 05:37:30.168997 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb84e953-49b4-454f-92de-60c7959ba6cf admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:37:30.176381 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-cb84e953-49b4-454f-92de-60c7959ba6cf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:37:30.253548 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed87111-2894-4d66-bf28-82a51a5e9899 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:37:30.254207 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed87111-2894-4d66-bf28-82a51a5e9899 admin admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:37:30.254495 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed87111-2894-4d66-bf28-82a51a5e9899 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:37:30.254778 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed87111-2894-4d66-bf28-82a51a5e9899 admin admin] PATH_INFO: `/v3/domains` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:37:30.255740 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ed87111-2894-4d66-bf28-82a51a5e9899 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:37:30.255836 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ed87111-2894-4d66-bf28-82a51a5e9899 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:37:30.260496 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ed87111-2894-4d66-bf28-82a51a5e9899 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:37:30.289890 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 879/1772] 10.222.0.33 () {58 vars in 1152 bytes} [Mon Jul 25 05:37:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 284 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:37:30.295151 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0395956-1ac4-41ce-8bf6-4c1ff72c6c76 None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:37:30.297046 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0395956-1ac4-41ce-8bf6-4c1ff72c6c76 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:37:30.298416 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0395956-1ac4-41ce-8bf6-4c1ff72c6c76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:37:30.431889 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-b0395956-1ac4-41ce-8bf6-4c1ff72c6c76 None None] MFA Rules not processed for user `1cd482a61e8a420db45bb532e5dae6e2`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:37:30.433445 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-b0395956-1ac4-41ce-8bf6-4c1ff72c6c76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:37:30.481986 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb84e953-49b4-454f-92de-60c7959ba6cf admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:37:30.486956 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a22c6bb0-3379-4823-8ff2-6b7a98d44b59 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db57b478c25643dbaf936f90f6e17d0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92afba266cce4d1bad4134c351e74be3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:37:30.487656 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22c6bb0-3379-4823-8ff2-6b7a98d44b59 service cinder] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:37:30.487992 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22c6bb0-3379-4823-8ff2-6b7a98d44b59 service cinder] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:37:30.488242 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22c6bb0-3379-4823-8ff2-6b7a98d44b59 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:37:30.491387 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-a22c6bb0-3379-4823-8ff2-6b7a98d44b59 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:37:30.676401 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a22c6bb0-3379-4823-8ff2-6b7a98d44b59 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:37:30.676401 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a22c6bb0-3379-4823-8ff2-6b7a98d44b59 service cinder] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:37:30.694334 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 880/1773] 10.222.0.33 () {58 vars in 1018 bytes} [Mon Jul 25 05:37:30 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 404 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:37:30.699845 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0395956-1ac4-41ce-8bf6-4c1ff72c6c76 None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:37:30.707227 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-b0395956-1ac4-41ce-8bf6-4c1ff72c6c76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:37:30.745795 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0395956-1ac4-41ce-8bf6-4c1ff72c6c76 None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:37:30.750371 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 894/1774] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jul 25 05:37:30 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 592 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 05:37:30.756172 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a22c6bb0-3379-4823-8ff2-6b7a98d44b59 service cinder] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:37:30.762151 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-a22c6bb0-3379-4823-8ff2-6b7a98d44b59 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:37:30.764569 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de7d039-e356-4293-bd22-2ed5de3e248c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cd482a61e8a420db45bb532e5dae6e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92afba266cce4d1bad4134c351e74be3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:37:30.765334 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de7d039-e356-4293-bd22-2ed5de3e248c service nova] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:37:30.765630 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de7d039-e356-4293-bd22-2ed5de3e248c service nova] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:37:30.766042 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de7d039-e356-4293-bd22-2ed5de3e248c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:37:30.771062 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-9de7d039-e356-4293-bd22-2ed5de3e248c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:37:30.826709 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a22c6bb0-3379-4823-8ff2-6b7a98d44b59 service cinder] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:37:30.850676 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25cbdc80-14bf-41f8-9bde-139ddc8eb235 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:37:30.852278 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25cbdc80-14bf-41f8-9bde-139ddc8eb235 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:37:30.852773 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25cbdc80-14bf-41f8-9bde-139ddc8eb235 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:37:30.853334 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25cbdc80-14bf-41f8-9bde-139ddc8eb235 admin admin] PATH_INFO: `/v3/users/f49a05a465134ab58f90c2877f830207` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:37:30.857602 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25cbdc80-14bf-41f8-9bde-139ddc8eb235 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f49a05a465134ab58f90c2877f830207)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:37:30.860586 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25cbdc80-14bf-41f8-9bde-139ddc8eb235 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:37:30.967043 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9de7d039-e356-4293-bd22-2ed5de3e248c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:37:30.974597 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9de7d039-e356-4293-bd22-2ed5de3e248c service nova] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:37:31.005106 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-25cbdc80-14bf-41f8-9bde-139ddc8eb235 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f49a05a465134ab58f90c2877f830207'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:37:31.006015 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-25cbdc80-14bf-41f8-9bde-139ddc8eb235 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f49a05a465134ab58f90c2877f830207'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:37:31.027795 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 881/1775] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jul 25 05:37:30 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 333 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 05:37:31.032298 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de7d039-e356-4293-bd22-2ed5de3e248c service nova] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:37:31.033171 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-25cbdc80-14bf-41f8-9bde-139ddc8eb235 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f49a05a465134ab58f90c2877f830207'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:37:31.035292 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-9de7d039-e356-4293-bd22-2ed5de3e248c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:37:31.095029 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-25cbdc80-14bf-41f8-9bde-139ddc8eb235 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f49a05a465134ab58f90c2877f830207'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:37:31.099155 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-25cbdc80-14bf-41f8-9bde-139ddc8eb235 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f49a05a465134ab58f90c2877f830207'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:37:31.102042 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de7d039-e356-4293-bd22-2ed5de3e248c service nova] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:37:31.122249 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8140d7ce-f182-4aa6-b1de-bff456b72d72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:37:31.123953 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:DELETE=14,keystone:SELECT=84,keystone:INSERT=2 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:37:31.126741 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8140d7ce-f182-4aa6-b1de-bff456b72d72 admin admin] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:37:31.127681 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8140d7ce-f182-4aa6-b1de-bff456b72d72 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:37:31.130381 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8140d7ce-f182-4aa6-b1de-bff456b72d72 admin admin] PATH_INFO: `/v3/projects` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:37:31.131407 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8140d7ce-f182-4aa6-b1de-bff456b72d72 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:37:31.146903 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8140d7ce-f182-4aa6-b1de-bff456b72d72 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:37:31.162863 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 895/1776] 10.222.0.33 () {58 vars in 1223 bytes} [Mon Jul 25 05:37:30 2022] DELETE /identity/v3/users/f49a05a465134ab58f90c2877f830207 => generated 0 bytes in 409 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 05:37:31.162863 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25cbdc80-14bf-41f8-9bde-139ddc8eb235 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:37:31.169427 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-25cbdc80-14bf-41f8-9bde-139ddc8eb235 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:37:31.271718 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 882/1777] 10.222.0.33 () {60 vars in 1152 bytes} [Mon Jul 25 05:37:31 2022] POST /identity/v3/projects => generated 389 bytes in 243 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 05:37:31.276731 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8140d7ce-f182-4aa6-b1de-bff456b72d72 admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:37:31.287434 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-8140d7ce-f182-4aa6-b1de-bff456b72d72 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:37:31.354482 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:37:31.373120 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25cbdc80-14bf-41f8-9bde-139ddc8eb235 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:37:31.551144 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d687282d-ddf3-4ce3-b8ea-78523a60c77a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db57b478c25643dbaf936f90f6e17d0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92afba266cce4d1bad4134c351e74be3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:37:31.553801 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d687282d-ddf3-4ce3-b8ea-78523a60c77a service cinder] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:37:31.553801 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d687282d-ddf3-4ce3-b8ea-78523a60c77a service cinder] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:37:31.553801 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d687282d-ddf3-4ce3-b8ea-78523a60c77a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:37:31.556902 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-d687282d-ddf3-4ce3-b8ea-78523a60c77a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:37:31.582870 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8140d7ce-f182-4aa6-b1de-bff456b72d72 admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:37:31.598455 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d202244-6341-4303-a569-69ab181c16c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:37:31.599379 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d202244-6341-4303-a569-69ab181c16c0 admin admin] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:37:31.599812 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d202244-6341-4303-a569-69ab181c16c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:37:31.600226 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d202244-6341-4303-a569-69ab181c16c0 admin admin] PATH_INFO: `/v3/users` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:37:31.601108 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d202244-6341-4303-a569-69ab181c16c0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:37:31.620681 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d202244-6341-4303-a569-69ab181c16c0 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:37:31.681675 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 883/1778] 10.222.0.33 () {60 vars in 1143 bytes} [Mon Jul 25 05:37:31 2022] POST /identity/v3/users => generated 295 bytes in 406 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 05:37:31.682860 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d202244-6341-4303-a569-69ab181c16c0 admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:37:31.684560 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-2d202244-6341-4303-a569-69ab181c16c0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:37:31.727648 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d687282d-ddf3-4ce3-b8ea-78523a60c77a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:37:31.731373 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d687282d-ddf3-4ce3-b8ea-78523a60c77a service cinder] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:37:31.760004 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 896/1779] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jul 25 05:37:31 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 599 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 05:37:31.763866 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd223b1-237f-461b-a94d-9568f254450f None None] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:37:31.765396 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd223b1-237f-461b-a94d-9568f254450f None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:37:31.767561 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd223b1-237f-461b-a94d-9568f254450f None None] PATH_INFO: `/` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:37:31.771450 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 897/1780] 10.222.0.33 () {54 vars in 921 bytes} [Mon Jul 25 05:37:31 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:37:31.772886 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd223b1-237f-461b-a94d-9568f254450f None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:37:31.802849 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd223b1-237f-461b-a94d-9568f254450f None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:37:31.814047 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5e540e-b557-416d-ba98-722113511d3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:37:31.817082 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5e540e-b557-416d-ba98-722113511d3f admin admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:37:31.817628 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5e540e-b557-416d-ba98-722113511d3f admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:37:31.819232 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5e540e-b557-416d-ba98-722113511d3f admin admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:37:31.820297 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba5e540e-b557-416d-ba98-722113511d3f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:37:31.824638 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba5e540e-b557-416d-ba98-722113511d3f admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:37:31.861183 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 898/1781] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jul 25 05:37:31 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 05:37:31.862444 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5e540e-b557-416d-ba98-722113511d3f admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:37:31.887986 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5e540e-b557-416d-ba98-722113511d3f admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:37:31.898435 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd82adc-a3db-4a0c-a2ac-edd1de69f252 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:37:31.899322 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd82adc-a3db-4a0c-a2ac-edd1de69f252 admin admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:37:31.900381 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd82adc-a3db-4a0c-a2ac-edd1de69f252 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:37:31.902102 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd82adc-a3db-4a0c-a2ac-edd1de69f252 admin admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:37:31.904322 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cd82adc-a3db-4a0c-a2ac-edd1de69f252 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:37:31.907626 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cd82adc-a3db-4a0c-a2ac-edd1de69f252 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:37:31.928984 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d202244-6341-4303-a569-69ab181c16c0 admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:37:31.940543 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 899/1782] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jul 25 05:37:31 2022] GET /identity/v3/roles => generated 1420 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 05:37:31.943297 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf79e47-cdac-4319-bec8-e13c38e6e7f3 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:37:31.944762 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf79e47-cdac-4319-bec8-e13c38e6e7f3 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:37:31.946016 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf79e47-cdac-4319-bec8-e13c38e6e7f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:37:31.951715 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c26030b4-10e7-4eab-ac44-6eb8203e23a9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cd482a61e8a420db45bb532e5dae6e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92afba266cce4d1bad4134c351e74be3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:37:31.952805 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26030b4-10e7-4eab-ac44-6eb8203e23a9 service nova] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:37:31.953258 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26030b4-10e7-4eab-ac44-6eb8203e23a9 service nova] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:37:31.954561 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26030b4-10e7-4eab-ac44-6eb8203e23a9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:37:31.967885 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-c26030b4-10e7-4eab-ac44-6eb8203e23a9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:37:31.987213 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-adf79e47-cdac-4319-bec8-e13c38e6e7f3 None None] MFA Rules not processed for user `1cd482a61e8a420db45bb532e5dae6e2`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:37:31.989605 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-adf79e47-cdac-4319-bec8-e13c38e6e7f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:37:32.007768 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 900/1783] 10.222.0.33 () {58 vars in 1018 bytes} [Mon Jul 25 05:37:31 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 67 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:37:32.009060 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf79e47-cdac-4319-bec8-e13c38e6e7f3 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:37:32.019100 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c26030b4-10e7-4eab-ac44-6eb8203e23a9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:37:32.023793 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c26030b4-10e7-4eab-ac44-6eb8203e23a9 service nova] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:37:32.031650 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf79e47-cdac-4319-bec8-e13c38e6e7f3 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:37:32.045374 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a02cfa-3db2-429a-b8ef-ba107adf6cec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:37:32.049165 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a02cfa-3db2-429a-b8ef-ba107adf6cec admin admin] REQUEST_METHOD: `PUT` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:37:32.049643 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a02cfa-3db2-429a-b8ef-ba107adf6cec admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:37:32.050032 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a02cfa-3db2-429a-b8ef-ba107adf6cec admin admin] PATH_INFO: `/v3/projects/d7a08dc773bd4cf59652c998b97752c7/users/8fd48c1cb0e1464889daa61783d4f268/roles/5074e74e35f54d4b9bb415c34999a82a` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:37:32.076747 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 884/1784] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jul 25 05:37:31 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 394 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 05:37:32.097146 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61a02cfa-3db2-429a-b8ef-ba107adf6cec admin admin] RBAC: Authorizing `identity:create_grant(project_id=d7a08dc773bd4cf59652c998b97752c7, user_id=8fd48c1cb0e1464889daa61783d4f268, role_id=5074e74e35f54d4b9bb415c34999a82a)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:37:32.106036 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61a02cfa-3db2-429a-b8ef-ba107adf6cec admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:37:32.144517 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 901/1785] 10.222.0.33 () {60 vars in 1482 bytes} [Mon Jul 25 05:37:32 2022] PUT /identity/v3/projects/d7a08dc773bd4cf59652c998b97752c7/users/8fd48c1cb0e1464889daa61783d4f268/roles/5074e74e35f54d4b9bb415c34999a82a => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 05:37:32.155337 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c6ed4a-3582-4390-a69e-479b5ab297db None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:37:32.155652 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c6ed4a-3582-4390-a69e-479b5ab297db None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:37:32.155935 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c6ed4a-3582-4390-a69e-479b5ab297db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:37:32.272292 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-d9c6ed4a-3582-4390-a69e-479b5ab297db None None] MFA Rules not processed for user `8fd48c1cb0e1464889daa61783d4f268`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:37:32.279532 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-d9c6ed4a-3582-4390-a69e-479b5ab297db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:37:32.561867 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 885/1786] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 25 05:37:32 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 408 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:37:32.566831 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a02cfa-3db2-429a-b8ef-ba107adf6cec admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:37:32.598240 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:37:32.674000 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c6ed4a-3582-4390-a69e-479b5ab297db None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:37:32.747063 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:37:32.879496 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a02cfa-3db2-429a-b8ef-ba107adf6cec admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:37:32.901574 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e099486-27ad-479a-9e02-e64064a9d9ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:37:32.902276 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e099486-27ad-479a-9e02-e64064a9d9ba admin admin] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:37:32.902563 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e099486-27ad-479a-9e02-e64064a9d9ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:37:32.902844 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e099486-27ad-479a-9e02-e64064a9d9ba admin admin] PATH_INFO: `/v3/projects` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:37:32.903551 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e099486-27ad-479a-9e02-e64064a9d9ba admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:37:32.905503 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e099486-27ad-479a-9e02-e64064a9d9ba admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:37:32.922873 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c6ed4a-3582-4390-a69e-479b5ab297db None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:37:32.929726 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ff66e6-b364-4d20-90c6-9cabbe1527ee service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cd482a61e8a420db45bb532e5dae6e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92afba266cce4d1bad4134c351e74be3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:37:32.930562 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ff66e6-b364-4d20-90c6-9cabbe1527ee service nova] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:37:32.930852 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ff66e6-b364-4d20-90c6-9cabbe1527ee service nova] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:37:32.931334 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ff66e6-b364-4d20-90c6-9cabbe1527ee service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:37:32.946366 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8ff66e6-b364-4d20-90c6-9cabbe1527ee service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:37:32.949035 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8ff66e6-b364-4d20-90c6-9cabbe1527ee service nova] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:37:32.966921 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 886/1787] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jul 25 05:37:32 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 293 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 25 05:37:33.013717 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 902/1788] 10.222.0.33 () {60 vars in 1152 bytes} [Mon Jul 25 05:37:32 2022] POST /identity/v3/projects => generated 389 bytes in 447 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 05:37:33.018599 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ff66e6-b364-4d20-90c6-9cabbe1527ee service nova] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:37:33.022830 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e099486-27ad-479a-9e02-e64064a9d9ba admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:37:33.024505 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-8e099486-27ad-479a-9e02-e64064a9d9ba admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:37:33.042779 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ff66e6-b364-4d20-90c6-9cabbe1527ee service nova] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:37:33.171346 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e099486-27ad-479a-9e02-e64064a9d9ba admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:37:33.185929 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ecebc7c-9ce1-479c-8655-aea52162a947 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:37:33.186842 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ecebc7c-9ce1-479c-8655-aea52162a947 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:37:33.187130 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ecebc7c-9ce1-479c-8655-aea52162a947 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:37:33.187482 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ecebc7c-9ce1-479c-8655-aea52162a947 admin admin] PATH_INFO: `/v3/users/a8ed129b857a4aff954e16c1d2ec29ae` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:37:33.188913 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ecebc7c-9ce1-479c-8655-aea52162a947 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a8ed129b857a4aff954e16c1d2ec29ae)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:37:33.190848 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ecebc7c-9ce1-479c-8655-aea52162a947 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:37:33.193617 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e903738-8d71-4762-ac1b-58354c6baf7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:37:33.194618 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e903738-8d71-4762-ac1b-58354c6baf7d admin admin] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:37:33.194977 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e903738-8d71-4762-ac1b-58354c6baf7d admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:37:33.195191 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e903738-8d71-4762-ac1b-58354c6baf7d admin admin] PATH_INFO: `/v3/users` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:37:33.195926 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e903738-8d71-4762-ac1b-58354c6baf7d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:37:33.200736 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e903738-8d71-4762-ac1b-58354c6baf7d admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:37:33.272521 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 887/1789] 10.222.0.33 () {60 vars in 1143 bytes} [Mon Jul 25 05:37:33 2022] POST /identity/v3/users => generated 301 bytes in 255 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 05:37:33.279762 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e903738-8d71-4762-ac1b-58354c6baf7d admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:37:33.314708 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-4ecebc7c-9ce1-479c-8655-aea52162a947 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a8ed129b857a4aff954e16c1d2ec29ae'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:37:33.315146 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-4ecebc7c-9ce1-479c-8655-aea52162a947 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a8ed129b857a4aff954e16c1d2ec29ae'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:37:33.328467 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-4ecebc7c-9ce1-479c-8655-aea52162a947 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a8ed129b857a4aff954e16c1d2ec29ae'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:37:33.332391 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e903738-8d71-4762-ac1b-58354c6baf7d admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:37:33.355349 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a211b68-d76c-48fc-9d7b-6faaadbc72d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:37:33.357375 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a211b68-d76c-48fc-9d7b-6faaadbc72d0 admin admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:37:33.357757 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a211b68-d76c-48fc-9d7b-6faaadbc72d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:37:33.358026 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a211b68-d76c-48fc-9d7b-6faaadbc72d0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:37:33.359057 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-4ecebc7c-9ce1-479c-8655-aea52162a947 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a8ed129b857a4aff954e16c1d2ec29ae'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:37:33.363544 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a211b68-d76c-48fc-9d7b-6faaadbc72d0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:37:33.364128 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-4ecebc7c-9ce1-479c-8655-aea52162a947 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a8ed129b857a4aff954e16c1d2ec29ae'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:37:33.364594 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a211b68-d76c-48fc-9d7b-6faaadbc72d0 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:37:33.393727 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 903/1790] 10.222.0.33 () {58 vars in 1223 bytes} [Mon Jul 25 05:37:33 2022] DELETE /identity/v3/users/a8ed129b857a4aff954e16c1d2ec29ae => generated 0 bytes in 371 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 05:37:33.397992 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0735347c-8c36-46f7-9d26-2bf0f040abe0 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:37:33.398287 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0735347c-8c36-46f7-9d26-2bf0f040abe0 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:37:33.398565 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0735347c-8c36-46f7-9d26-2bf0f040abe0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:37:33.402848 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 888/1791] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jul 25 05:37:33 2022] GET /identity/v3/roles => generated 1420 bytes in 123 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 05:37:33.408496 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a211b68-d76c-48fc-9d7b-6faaadbc72d0 admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:37:33.416254 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-6a211b68-d76c-48fc-9d7b-6faaadbc72d0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:37:33.435745 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-0735347c-8c36-46f7-9d26-2bf0f040abe0 None None] MFA Rules not processed for user `1eae540242d94c10b0ad33960a855f08`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:37:33.440946 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-0735347c-8c36-46f7-9d26-2bf0f040abe0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:37:33.450584 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a211b68-d76c-48fc-9d7b-6faaadbc72d0 admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:37:33.479607 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:37:33.646843 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 904/1792] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 25 05:37:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 247 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 05:37:33.690494 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0735347c-8c36-46f7-9d26-2bf0f040abe0 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:37:33.692751 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-0735347c-8c36-46f7-9d26-2bf0f040abe0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:37:33.752465 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c3fa2cb-7156-4843-adc9-ca186198e25c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:37:33.754329 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3fa2cb-7156-4843-adc9-ca186198e25c admin admin] REQUEST_METHOD: `PUT` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:37:33.757525 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3fa2cb-7156-4843-adc9-ca186198e25c admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:37:33.761120 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3fa2cb-7156-4843-adc9-ca186198e25c admin admin] PATH_INFO: `/v3/projects/9a4d72eaabc943c598bf30d7b744e5f1/users/1f3ca7d62dca4857adce7a350e388828/roles/39e5c666285a42fe8fc2c3afca6260d1` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:37:33.854567 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c3fa2cb-7156-4843-adc9-ca186198e25c admin admin] RBAC: Authorizing `identity:create_grant(project_id=9a4d72eaabc943c598bf30d7b744e5f1, user_id=1f3ca7d62dca4857adce7a350e388828, role_id=39e5c666285a42fe8fc2c3afca6260d1)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:37:33.874275 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0735347c-8c36-46f7-9d26-2bf0f040abe0 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:37:33.874564 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c3fa2cb-7156-4843-adc9-ca186198e25c admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:37:33.892134 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f66daa8f-5509-403d-a1a0-0360504a3188 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b2931200f99e43d79a50a9f6e02738e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92afba266cce4d1bad4134c351e74be3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:37:33.893915 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66daa8f-5509-403d-a1a0-0360504a3188 service neutron] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:37:33.893915 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66daa8f-5509-403d-a1a0-0360504a3188 service neutron] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:37:33.894089 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66daa8f-5509-403d-a1a0-0360504a3188 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:37:33.905716 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 889/1793] 10.222.0.33 () {60 vars in 1482 bytes} [Mon Jul 25 05:37:33 2022] PUT /identity/v3/projects/9a4d72eaabc943c598bf30d7b744e5f1/users/1f3ca7d62dca4857adce7a350e388828/roles/39e5c666285a42fe8fc2c3afca6260d1 => generated 0 bytes in 495 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 05:37:33.906812 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c3fa2cb-7156-4843-adc9-ca186198e25c admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:37:33.923762 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f66daa8f-5509-403d-a1a0-0360504a3188 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:37:33.925830 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:37:33.927746 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f66daa8f-5509-403d-a1a0-0360504a3188 service neutron] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:37:33.966918 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 905/1794] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jul 25 05:37:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 277 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 05:37:34.140896 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c3fa2cb-7156-4843-adc9-ca186198e25c admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:37:34.171191 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b732a3d-dde4-443e-a238-5c50af53074c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:37:34.172089 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b732a3d-dde4-443e-a238-5c50af53074c admin admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:37:34.172566 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b732a3d-dde4-443e-a238-5c50af53074c admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:37:34.173117 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b732a3d-dde4-443e-a238-5c50af53074c admin admin] PATH_INFO: `/v3/roles` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:37:34.174056 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b732a3d-dde4-443e-a238-5c50af53074c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:37:34.176690 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b732a3d-dde4-443e-a238-5c50af53074c admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:37:34.222420 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 890/1795] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jul 25 05:37:33 2022] GET /identity/v3/roles => generated 1420 bytes in 316 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 05:37:34.228626 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f66daa8f-5509-403d-a1a0-0360504a3188 service neutron] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:37:34.253852 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f66daa8f-5509-403d-a1a0-0360504a3188 service neutron] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:37:34.280581 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54983518-69a5-4524-8315-ea1ab5669f22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:37:34.289793 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54983518-69a5-4524-8315-ea1ab5669f22 admin admin] REQUEST_METHOD: `PUT` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:37:34.291373 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54983518-69a5-4524-8315-ea1ab5669f22 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:37:34.292455 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54983518-69a5-4524-8315-ea1ab5669f22 admin admin] PATH_INFO: `/v3/projects/9a4d72eaabc943c598bf30d7b744e5f1/users/1f3ca7d62dca4857adce7a350e388828/roles/39e5c666285a42fe8fc2c3afca6260d1` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:37:34.304018 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54983518-69a5-4524-8315-ea1ab5669f22 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9a4d72eaabc943c598bf30d7b744e5f1, user_id=1f3ca7d62dca4857adce7a350e388828, role_id=39e5c666285a42fe8fc2c3afca6260d1)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:37:34.312838 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54983518-69a5-4524-8315-ea1ab5669f22 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:37:34.346935 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 906/1796] 10.222.0.33 () {60 vars in 1482 bytes} [Mon Jul 25 05:37:34 2022] PUT /identity/v3/projects/9a4d72eaabc943c598bf30d7b744e5f1/users/1f3ca7d62dca4857adce7a350e388828/roles/39e5c666285a42fe8fc2c3afca6260d1 => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 05:37:34.358871 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc64c4d-62d4-41ce-8ddf-9cfd5cfcbc6f None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:37:34.359062 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc64c4d-62d4-41ce-8ddf-9cfd5cfcbc6f None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:37:34.359404 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc64c4d-62d4-41ce-8ddf-9cfd5cfcbc6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:37:34.420884 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-3fc64c4d-62d4-41ce-8ddf-9cfd5cfcbc6f None None] MFA Rules not processed for user `1f3ca7d62dca4857adce7a350e388828`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:37:34.425004 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-3fc64c4d-62d4-41ce-8ddf-9cfd5cfcbc6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:37:34.634757 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 891/1797] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 25 05:37:34 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 277 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:37:34.655018 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54983518-69a5-4524-8315-ea1ab5669f22 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:37:34.657352 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-54983518-69a5-4524-8315-ea1ab5669f22 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:37:34.831634 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54983518-69a5-4524-8315-ea1ab5669f22 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:37:34.841290 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc79a12c-33cc-4eae-a8ef-c724a11779eb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db57b478c25643dbaf936f90f6e17d0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92afba266cce4d1bad4134c351e74be3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:37:34.842324 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc79a12c-33cc-4eae-a8ef-c724a11779eb service cinder] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:37:34.842829 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc79a12c-33cc-4eae-a8ef-c724a11779eb service cinder] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:37:34.843284 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc79a12c-33cc-4eae-a8ef-c724a11779eb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:37:34.859183 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc79a12c-33cc-4eae-a8ef-c724a11779eb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:37:34.861191 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc79a12c-33cc-4eae-a8ef-c724a11779eb service cinder] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:37:34.899796 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 907/1798] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jul 25 05:37:34 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 248 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 05:37:35.167260 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc64c4d-62d4-41ce-8ddf-9cfd5cfcbc6f None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:37:35.169469 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-3fc64c4d-62d4-41ce-8ddf-9cfd5cfcbc6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:37:35.216111 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc79a12c-33cc-4eae-a8ef-c724a11779eb service cinder] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:37:35.409420 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc64c4d-62d4-41ce-8ddf-9cfd5cfcbc6f None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:37:35.439088 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4977c0be-f4ca-4d86-945e-dac89fa6ab47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:37:35.441518 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4977c0be-f4ca-4d86-945e-dac89fa6ab47 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:37:35.443040 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4977c0be-f4ca-4d86-945e-dac89fa6ab47 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:37:35.444304 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4977c0be-f4ca-4d86-945e-dac89fa6ab47 admin admin] PATH_INFO: `/v3/projects/f8deb340b1b14caab9fa42b11064d309` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:37:35.447525 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4977c0be-f4ca-4d86-945e-dac89fa6ab47 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f8deb340b1b14caab9fa42b11064d309)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:37:35.453601 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4977c0be-f4ca-4d86-945e-dac89fa6ab47 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:37:35.473440 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:37:35.537837 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc79a12c-33cc-4eae-a8ef-c724a11779eb service cinder] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:37:35.556174 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef04e3a3-d8c4-497a-808c-144249b41682 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:37:35.562859 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef04e3a3-d8c4-497a-808c-144249b41682 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:37:35.563345 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef04e3a3-d8c4-497a-808c-144249b41682 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:37:35.563494 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef04e3a3-d8c4-497a-808c-144249b41682 admin admin] PATH_INFO: `/v3/users/8fd48c1cb0e1464889daa61783d4f268` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:37:35.572601 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef04e3a3-d8c4-497a-808c-144249b41682 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8fd48c1cb0e1464889daa61783d4f268)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:37:35.580351 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:37:35.580836 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef04e3a3-d8c4-497a-808c-144249b41682 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:37:35.625054 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-4977c0be-f4ca-4d86-945e-dac89fa6ab47 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f8deb340b1b14caab9fa42b11064d309', 'request_id': 'req-4977c0be-f4ca-4d86-945e-dac89fa6ab47'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:37:35.632979 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-4977c0be-f4ca-4d86-945e-dac89fa6ab47 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f8deb340b1b14caab9fa42b11064d309', 'request_id': 'req-4977c0be-f4ca-4d86-945e-dac89fa6ab47'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:37:35.647703 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-4977c0be-f4ca-4d86-945e-dac89fa6ab47 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f8deb340b1b14caab9fa42b11064d309', 'request_id': 'req-4977c0be-f4ca-4d86-945e-dac89fa6ab47'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:37:35.693627 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-4977c0be-f4ca-4d86-945e-dac89fa6ab47 admin admin] The token cache is being invalidate because project f8deb340b1b14caab9fa42b11064d309 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80248) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 05:37:35.694269 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-4977c0be-f4ca-4d86-945e-dac89fa6ab47 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:37:35.701954 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-4977c0be-f4ca-4d86-945e-dac89fa6ab47 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:37:35.703711 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 892/1799] 10.222.0.33 () {58 vars in 1232 bytes} [Mon Jul 25 05:37:35 2022] DELETE /identity/v3/projects/f8deb340b1b14caab9fa42b11064d309 => generated 0 bytes in 536 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 05:37:35.709515 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a035a0-45a4-427d-a2c5-44c882933ce1 None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:37:35.709810 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a035a0-45a4-427d-a2c5-44c882933ce1 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:37:35.710131 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a035a0-45a4-427d-a2c5-44c882933ce1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:37:35.749720 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-ef04e3a3-d8c4-497a-808c-144249b41682 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8fd48c1cb0e1464889daa61783d4f268'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:37:35.750593 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-ef04e3a3-d8c4-497a-808c-144249b41682 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8fd48c1cb0e1464889daa61783d4f268'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:37:35.759838 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-ef04e3a3-d8c4-497a-808c-144249b41682 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8fd48c1cb0e1464889daa61783d4f268'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:37:35.766659 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-15a035a0-45a4-427d-a2c5-44c882933ce1 None None] MFA Rules not processed for user `1eae540242d94c10b0ad33960a855f08`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:37:35.767898 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-15a035a0-45a4-427d-a2c5-44c882933ce1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:37:35.793558 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-ef04e3a3-d8c4-497a-808c-144249b41682 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8fd48c1cb0e1464889daa61783d4f268'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:37:35.795118 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-ef04e3a3-d8c4-497a-808c-144249b41682 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8fd48c1cb0e1464889daa61783d4f268'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:37:35.827935 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 908/1800] 10.222.0.33 () {58 vars in 1223 bytes} [Mon Jul 25 05:37:35 2022] DELETE /identity/v3/users/8fd48c1cb0e1464889daa61783d4f268 => generated 0 bytes in 610 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 05:37:35.827935 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef04e3a3-d8c4-497a-808c-144249b41682 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:37:35.833341 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-ef04e3a3-d8c4-497a-808c-144249b41682 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:37:36.004691 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 893/1801] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 25 05:37:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 296 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:37:36.010016 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef04e3a3-d8c4-497a-808c-144249b41682 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:37:36.010539 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25954fe1-bd1a-4364-886d-5fbdaf5ca269 None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:37:36.011072 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25954fe1-bd1a-4364-886d-5fbdaf5ca269 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:37:36.011503 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25954fe1-bd1a-4364-886d-5fbdaf5ca269 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:37:36.024277 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f8df21-33d7-43fe-acd6-32de45b77c0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:37:36.028029 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f8df21-33d7-43fe-acd6-32de45b77c0b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:37:36.028351 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f8df21-33d7-43fe-acd6-32de45b77c0b admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:37:36.028645 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f8df21-33d7-43fe-acd6-32de45b77c0b admin admin] PATH_INFO: `/v3/users/b0fba4dd135043998a2e3a4e8837d10d` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:37:36.030166 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4f8df21-33d7-43fe-acd6-32de45b77c0b admin admin] RBAC: Authorizing `identity:delete_user(user_id=b0fba4dd135043998a2e3a4e8837d10d)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:37:36.036127 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4f8df21-33d7-43fe-acd6-32de45b77c0b admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:37:36.042775 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-25954fe1-bd1a-4364-886d-5fbdaf5ca269 None None] MFA Rules not processed for user `1eae540242d94c10b0ad33960a855f08`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:37:36.044817 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-25954fe1-bd1a-4364-886d-5fbdaf5ca269 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:37:36.070094 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 894/1802] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 25 05:37:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 65 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:37:36.071712 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530461cf-a369-4dbd-b0d3-6987004818e7 None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:37:36.072129 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530461cf-a369-4dbd-b0d3-6987004818e7 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:37:36.072544 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530461cf-a369-4dbd-b0d3-6987004818e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:37:36.113035 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-530461cf-a369-4dbd-b0d3-6987004818e7 None None] MFA Rules not processed for user `1eae540242d94c10b0ad33960a855f08`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:37:36.114625 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-530461cf-a369-4dbd-b0d3-6987004818e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:37:36.146512 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 895/1803] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 25 05:37:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:37:36.147633 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-530461cf-a369-4dbd-b0d3-6987004818e7 None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:37:36.147748 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-f4f8df21-33d7-43fe-acd6-32de45b77c0b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b0fba4dd135043998a2e3a4e8837d10d'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:37:36.148430 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-f4f8df21-33d7-43fe-acd6-32de45b77c0b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b0fba4dd135043998a2e3a4e8837d10d'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:37:36.162852 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-f4f8df21-33d7-43fe-acd6-32de45b77c0b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b0fba4dd135043998a2e3a4e8837d10d'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:37:36.167712 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-530461cf-a369-4dbd-b0d3-6987004818e7 None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:37:36.199643 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=80,keystone:DELETE=19,keystone:INSERT=1 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:37:36.202943 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cbddaec-2b7e-4260-9527-13f1ccfd3b50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:37:36.204070 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cbddaec-2b7e-4260-9527-13f1ccfd3b50 admin admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:37:36.204718 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cbddaec-2b7e-4260-9527-13f1ccfd3b50 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:37:36.205365 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cbddaec-2b7e-4260-9527-13f1ccfd3b50 admin admin] PATH_INFO: `/v3/domains` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:37:36.205980 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cbddaec-2b7e-4260-9527-13f1ccfd3b50 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:37:36.206487 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cbddaec-2b7e-4260-9527-13f1ccfd3b50 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:37:36.208235 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cbddaec-2b7e-4260-9527-13f1ccfd3b50 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:37:36.226828 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-f4f8df21-33d7-43fe-acd6-32de45b77c0b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b0fba4dd135043998a2e3a4e8837d10d'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:37:36.230166 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-f4f8df21-33d7-43fe-acd6-32de45b77c0b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b0fba4dd135043998a2e3a4e8837d10d'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:37:36.244912 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 896/1804] 10.222.0.33 () {58 vars in 1152 bytes} [Mon Jul 25 05:37:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:37:36.247638 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cbddaec-2b7e-4260-9527-13f1ccfd3b50 admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:37:36.249578 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-4cbddaec-2b7e-4260-9527-13f1ccfd3b50 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:37:36.291929 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 909/1805] 10.222.0.33 () {58 vars in 1223 bytes} [Mon Jul 25 05:37:35 2022] DELETE /identity/v3/users/b0fba4dd135043998a2e3a4e8837d10d => generated 0 bytes in 466 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 05:37:36.300896 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319dba3d-a144-4154-a14c-9341e31359be None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:37:36.301080 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319dba3d-a144-4154-a14c-9341e31359be None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:37:36.301693 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319dba3d-a144-4154-a14c-9341e31359be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:37:36.352329 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cbddaec-2b7e-4260-9527-13f1ccfd3b50 admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:37:36.430077 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-319dba3d-a144-4154-a14c-9341e31359be None None] MFA Rules not processed for user `1eae540242d94c10b0ad33960a855f08`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:37:36.439743 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-319dba3d-a144-4154-a14c-9341e31359be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:37:36.638365 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e79abe2e-3bcd-491b-a5eb-70d34ed88d84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:37:36.638969 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79abe2e-3bcd-491b-a5eb-70d34ed88d84 admin admin] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:37:36.639263 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79abe2e-3bcd-491b-a5eb-70d34ed88d84 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:37:36.641886 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79abe2e-3bcd-491b-a5eb-70d34ed88d84 admin admin] PATH_INFO: `/v3/projects` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:37:36.647268 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e79abe2e-3bcd-491b-a5eb-70d34ed88d84 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:37:36.649457 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e79abe2e-3bcd-491b-a5eb-70d34ed88d84 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:37:36.731009 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 897/1806] 10.222.0.33 () {60 vars in 1152 bytes} [Mon Jul 25 05:37:36 2022] POST /identity/v3/projects => generated 387 bytes in 483 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 05:37:36.732665 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e79abe2e-3bcd-491b-a5eb-70d34ed88d84 admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:37:36.736925 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-e79abe2e-3bcd-491b-a5eb-70d34ed88d84 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:37:36.743428 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 910/1807] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 25 05:37:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 448 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:37:36.748936 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-319dba3d-a144-4154-a14c-9341e31359be None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:37:36.777812 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-319dba3d-a144-4154-a14c-9341e31359be None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:37:36.789611 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:DELETE=6,keystone:SELECT=92,keystone:INSERT=2 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:37:36.922742 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c6be95-25de-4e70-ac83-04f4ed6e5f51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:37:36.923833 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c6be95-25de-4e70-ac83-04f4ed6e5f51 admin admin] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:37:36.924584 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c6be95-25de-4e70-ac83-04f4ed6e5f51 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:37:36.924946 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c6be95-25de-4e70-ac83-04f4ed6e5f51 admin admin] PATH_INFO: `/v3/users` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:37:36.925779 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47c6be95-25de-4e70-ac83-04f4ed6e5f51 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:37:36.929732 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47c6be95-25de-4e70-ac83-04f4ed6e5f51 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:37:36.938108 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e79abe2e-3bcd-491b-a5eb-70d34ed88d84 admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:37:36.945076 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f64348c5-b902-4755-8345-3738668e2752 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b2931200f99e43d79a50a9f6e02738e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92afba266cce4d1bad4134c351e74be3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:37:36.945811 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64348c5-b902-4755-8345-3738668e2752 service neutron] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:37:36.946163 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64348c5-b902-4755-8345-3738668e2752 service neutron] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:37:36.946451 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64348c5-b902-4755-8345-3738668e2752 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:37:36.952031 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-f64348c5-b902-4755-8345-3738668e2752 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:37:36.975198 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 911/1808] 10.222.0.33 () {60 vars in 1143 bytes} [Mon Jul 25 05:37:36 2022] POST /identity/v3/users => generated 294 bytes in 227 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 05:37:36.976773 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c6be95-25de-4e70-ac83-04f4ed6e5f51 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:37:36.980117 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-47c6be95-25de-4e70-ac83-04f4ed6e5f51 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:37:36.998154 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c6be95-25de-4e70-ac83-04f4ed6e5f51 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:37:37.005225 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4609b7d7-d88a-4970-b9ad-056159d4cb70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:37:37.007998 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4609b7d7-d88a-4970-b9ad-056159d4cb70 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:37:37.008511 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4609b7d7-d88a-4970-b9ad-056159d4cb70 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:37:37.009046 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4609b7d7-d88a-4970-b9ad-056159d4cb70 admin admin] PATH_INFO: `/v3/users/59c9b2ed66a44f238b5908beec7638b8` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:37:37.011243 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f64348c5-b902-4755-8345-3738668e2752 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:37:37.011605 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4609b7d7-d88a-4970-b9ad-056159d4cb70 admin admin] RBAC: Authorizing `identity:delete_user(user_id=59c9b2ed66a44f238b5908beec7638b8)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:37:37.013926 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4609b7d7-d88a-4970-b9ad-056159d4cb70 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:37:37.016147 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f64348c5-b902-4755-8345-3738668e2752 service neutron] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:37:37.035982 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 898/1809] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jul 25 05:37:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 05:37:37.036342 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f64348c5-b902-4755-8345-3738668e2752 service neutron] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:37:37.063921 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f64348c5-b902-4755-8345-3738668e2752 service neutron] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:37:37.076485 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb485af-cae0-4b15-ace4-17402f4993ce service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b2931200f99e43d79a50a9f6e02738e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92afba266cce4d1bad4134c351e74be3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:37:37.077282 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb485af-cae0-4b15-ace4-17402f4993ce service neutron] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:37:37.078809 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb485af-cae0-4b15-ace4-17402f4993ce service neutron] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:37:37.079129 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb485af-cae0-4b15-ace4-17402f4993ce service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:37:37.097296 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fb485af-cae0-4b15-ace4-17402f4993ce service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:37:37.099258 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fb485af-cae0-4b15-ace4-17402f4993ce service neutron] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:37:37.122217 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 899/1810] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jul 25 05:37:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 05:37:37.123497 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb485af-cae0-4b15-ace4-17402f4993ce service neutron] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:37:37.129537 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-4fb485af-cae0-4b15-ace4-17402f4993ce service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:37:37.131099 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-4609b7d7-d88a-4970-b9ad-056159d4cb70 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '59c9b2ed66a44f238b5908beec7638b8'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:37:37.132397 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-4609b7d7-d88a-4970-b9ad-056159d4cb70 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '59c9b2ed66a44f238b5908beec7638b8'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:37:37.169607 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-4609b7d7-d88a-4970-b9ad-056159d4cb70 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '59c9b2ed66a44f238b5908beec7638b8'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:37:37.181586 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb485af-cae0-4b15-ace4-17402f4993ce service neutron] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:37:37.201356 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d018e0-517d-44c3-bb82-69ef3a4e3582 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:37:37.205062 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d018e0-517d-44c3-bb82-69ef3a4e3582 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:37:37.207882 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d018e0-517d-44c3-bb82-69ef3a4e3582 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:37:37.209920 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d018e0-517d-44c3-bb82-69ef3a4e3582 admin admin] PATH_INFO: `/v3/projects/65bae6dda085445d8ab9e7103b920e15` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:37:37.213400 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9d018e0-517d-44c3-bb82-69ef3a4e3582 admin admin] RBAC: Authorizing `identity:delete_project(project_id=65bae6dda085445d8ab9e7103b920e15)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:37:37.218326 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9d018e0-517d-44c3-bb82-69ef3a4e3582 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:37:37.219118 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-4609b7d7-d88a-4970-b9ad-056159d4cb70 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '59c9b2ed66a44f238b5908beec7638b8'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:37:37.220136 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-4609b7d7-d88a-4970-b9ad-056159d4cb70 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '59c9b2ed66a44f238b5908beec7638b8'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:37:37.224053 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:37:37.258503 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 912/1811] 10.222.0.33 () {58 vars in 1223 bytes} [Mon Jul 25 05:37:36 2022] DELETE /identity/v3/users/59c9b2ed66a44f238b5908beec7638b8 => generated 0 bytes in 279 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 05:37:37.258503 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4609b7d7-d88a-4970-b9ad-056159d4cb70 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:37:37.268006 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-4609b7d7-d88a-4970-b9ad-056159d4cb70 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:37:37.331912 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-b9d018e0-517d-44c3-bb82-69ef3a4e3582 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '65bae6dda085445d8ab9e7103b920e15', 'request_id': 'req-b9d018e0-517d-44c3-bb82-69ef3a4e3582'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:37:37.343316 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-b9d018e0-517d-44c3-bb82-69ef3a4e3582 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '65bae6dda085445d8ab9e7103b920e15', 'request_id': 'req-b9d018e0-517d-44c3-bb82-69ef3a4e3582'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:37:37.356204 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-b9d018e0-517d-44c3-bb82-69ef3a4e3582 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '65bae6dda085445d8ab9e7103b920e15', 'request_id': 'req-b9d018e0-517d-44c3-bb82-69ef3a4e3582'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:37:37.385420 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-b9d018e0-517d-44c3-bb82-69ef3a4e3582 admin admin] The token cache is being invalidate because project 65bae6dda085445d8ab9e7103b920e15 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80248) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 05:37:37.385651 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-b9d018e0-517d-44c3-bb82-69ef3a4e3582 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:37:37.388732 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-b9d018e0-517d-44c3-bb82-69ef3a4e3582 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:37:37.391180 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 900/1812] 10.222.0.33 () {58 vars in 1232 bytes} [Mon Jul 25 05:37:37 2022] DELETE /identity/v3/projects/65bae6dda085445d8ab9e7103b920e15 => generated 0 bytes in 268 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 05:37:37.392575 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa4950a-e773-43a2-b300-8bd06cd1c1e0 None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:37:37.392910 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa4950a-e773-43a2-b300-8bd06cd1c1e0 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:37:37.393208 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa4950a-e773-43a2-b300-8bd06cd1c1e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:37:37.438648 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-dfa4950a-e773-43a2-b300-8bd06cd1c1e0 None None] MFA Rules not processed for user `1eae540242d94c10b0ad33960a855f08`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:37:37.441618 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-dfa4950a-e773-43a2-b300-8bd06cd1c1e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:37:37.451618 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4609b7d7-d88a-4970-b9ad-056159d4cb70 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:37:37.677430 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61cc8452-dd52-41bb-9e5a-c7cb58a702ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:37:37.679249 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61cc8452-dd52-41bb-9e5a-c7cb58a702ee admin admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:37:37.679598 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61cc8452-dd52-41bb-9e5a-c7cb58a702ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:37:37.681088 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61cc8452-dd52-41bb-9e5a-c7cb58a702ee admin admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:37:37.682016 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61cc8452-dd52-41bb-9e5a-c7cb58a702ee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:37:37.683732 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 901/1813] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 25 05:37:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 292 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:37:37.687112 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61cc8452-dd52-41bb-9e5a-c7cb58a702ee admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:37:37.688385 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa4950a-e773-43a2-b300-8bd06cd1c1e0 None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:37:37.690117 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-dfa4950a-e773-43a2-b300-8bd06cd1c1e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:37:37.697409 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 913/1814] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jul 25 05:37:37 2022] GET /identity/v3/roles => generated 1420 bytes in 441 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 05:37:37.699542 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b209ae-67b0-4b98-80c6-d9190335d3b1 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:37:37.700194 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b209ae-67b0-4b98-80c6-d9190335d3b1 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:37:37.700495 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b209ae-67b0-4b98-80c6-d9190335d3b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:37:37.723587 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-f8b209ae-67b0-4b98-80c6-d9190335d3b1 None None] MFA Rules not processed for user `69f3276517de4f889dddd3180436ede9`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:37:37.724886 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-f8b209ae-67b0-4b98-80c6-d9190335d3b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:37:37.743077 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa4950a-e773-43a2-b300-8bd06cd1c1e0 None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:37:37.756754 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f52ea976-c7c5-4fac-b371-3c193a987045 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:37:37.757862 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52ea976-c7c5-4fac-b371-3c193a987045 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:37:37.758266 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52ea976-c7c5-4fac-b371-3c193a987045 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:37:37.758617 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52ea976-c7c5-4fac-b371-3c193a987045 admin admin] PATH_INFO: `/v3/users/8b6883c886294f6f9fe5dc09b2784ff2` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:37:37.759948 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f52ea976-c7c5-4fac-b371-3c193a987045 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8b6883c886294f6f9fe5dc09b2784ff2)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:37:37.763865 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f52ea976-c7c5-4fac-b371-3c193a987045 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:37:37.796128 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 914/1815] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 25 05:37:37 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:37:37.796128 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b209ae-67b0-4b98-80c6-d9190335d3b1 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:37:37.798193 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-f8b209ae-67b0-4b98-80c6-d9190335d3b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:37:37.798959 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:37:37.837200 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-f52ea976-c7c5-4fac-b371-3c193a987045 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8b6883c886294f6f9fe5dc09b2784ff2'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:37:37.852050 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:INSERT=1,keystone:DELETE=1,keystone:SELECT=98 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:37:37.852467 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-f52ea976-c7c5-4fac-b371-3c193a987045 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8b6883c886294f6f9fe5dc09b2784ff2'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:37:37.854650 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-f52ea976-c7c5-4fac-b371-3c193a987045 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8b6883c886294f6f9fe5dc09b2784ff2'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:37:37.864705 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-f52ea976-c7c5-4fac-b371-3c193a987045 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8b6883c886294f6f9fe5dc09b2784ff2'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:37:37.865365 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-f52ea976-c7c5-4fac-b371-3c193a987045 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8b6883c886294f6f9fe5dc09b2784ff2'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:37:37.869966 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b209ae-67b0-4b98-80c6-d9190335d3b1 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:37:37.874837 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd3f2b2-cce3-43ff-a0db-76cb77790f1d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b2931200f99e43d79a50a9f6e02738e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92afba266cce4d1bad4134c351e74be3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:37:37.875617 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd3f2b2-cce3-43ff-a0db-76cb77790f1d service neutron] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:37:37.876194 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd3f2b2-cce3-43ff-a0db-76cb77790f1d service neutron] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:37:37.876608 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd3f2b2-cce3-43ff-a0db-76cb77790f1d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:37:37.879210 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-dbd3f2b2-cce3-43ff-a0db-76cb77790f1d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:37:37.881081 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 902/1816] 10.222.0.33 () {58 vars in 1223 bytes} [Mon Jul 25 05:37:37 2022] DELETE /identity/v3/users/8b6883c886294f6f9fe5dc09b2784ff2 => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 05:37:37.886461 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f52ea976-c7c5-4fac-b371-3c193a987045 admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:37:37.888359 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-f52ea976-c7c5-4fac-b371-3c193a987045 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:37:37.895413 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbd3f2b2-cce3-43ff-a0db-76cb77790f1d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:37:37.897595 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbd3f2b2-cce3-43ff-a0db-76cb77790f1d service neutron] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:37:37.911283 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 915/1817] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jul 25 05:37:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 05:37:37.977070 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f52ea976-c7c5-4fac-b371-3c193a987045 admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:37:37.983156 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12bd1e00-a8d7-4535-b521-c55ac2873e8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:37:37.983858 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12bd1e00-a8d7-4535-b521-c55ac2873e8e admin admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:37:37.984640 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12bd1e00-a8d7-4535-b521-c55ac2873e8e admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:37:37.984640 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12bd1e00-a8d7-4535-b521-c55ac2873e8e admin admin] PATH_INFO: `/v3/roles` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:37:37.985217 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12bd1e00-a8d7-4535-b521-c55ac2873e8e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:37:37.986848 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12bd1e00-a8d7-4535-b521-c55ac2873e8e admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:37:38.011528 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 903/1818] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jul 25 05:37:37 2022] GET /identity/v3/roles => generated 1420 bytes in 119 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 05:37:38.011744 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd3f2b2-cce3-43ff-a0db-76cb77790f1d service neutron] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:37:38.035683 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd3f2b2-cce3-43ff-a0db-76cb77790f1d service neutron] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:37:38.046962 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e57b5f60-27c0-4f53-bfce-8ce578950581 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:37:38.048560 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57b5f60-27c0-4f53-bfce-8ce578950581 admin admin] REQUEST_METHOD: `PUT` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:37:38.049078 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57b5f60-27c0-4f53-bfce-8ce578950581 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:37:38.049738 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57b5f60-27c0-4f53-bfce-8ce578950581 admin admin] PATH_INFO: `/v3/projects/8dc782bcea24493e96f22f968c2b8e6c/users/69dcc7bdac564a4dbba23efe6d762b58/roles/5074e74e35f54d4b9bb415c34999a82a` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:37:38.086154 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e57b5f60-27c0-4f53-bfce-8ce578950581 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8dc782bcea24493e96f22f968c2b8e6c, user_id=69dcc7bdac564a4dbba23efe6d762b58, role_id=5074e74e35f54d4b9bb415c34999a82a)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:37:38.090071 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e57b5f60-27c0-4f53-bfce-8ce578950581 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:37:38.108138 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 916/1819] 10.222.0.33 () {60 vars in 1482 bytes} [Mon Jul 25 05:37:38 2022] PUT /identity/v3/projects/8dc782bcea24493e96f22f968c2b8e6c/users/69dcc7bdac564a4dbba23efe6d762b58/roles/5074e74e35f54d4b9bb415c34999a82a => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 05:37:38.124262 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614d830c-c304-4ac8-8a58-a8d77c3df719 None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:37:38.124602 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614d830c-c304-4ac8-8a58-a8d77c3df719 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:37:38.124875 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614d830c-c304-4ac8-8a58-a8d77c3df719 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:37:38.154975 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-614d830c-c304-4ac8-8a58-a8d77c3df719 None None] MFA Rules not processed for user `69dcc7bdac564a4dbba23efe6d762b58`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:37:38.156223 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-614d830c-c304-4ac8-8a58-a8d77c3df719 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:37:38.243285 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 904/1820] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 25 05:37:38 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:37:38.249934 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e57b5f60-27c0-4f53-bfce-8ce578950581 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:37:38.330081 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e57b5f60-27c0-4f53-bfce-8ce578950581 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:37:38.338299 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d957723d-e225-4d3d-b54f-b53ccf1a41a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:37:38.340035 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d957723d-e225-4d3d-b54f-b53ccf1a41a3 admin admin] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:37:38.340921 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d957723d-e225-4d3d-b54f-b53ccf1a41a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:37:38.342168 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d957723d-e225-4d3d-b54f-b53ccf1a41a3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:37:38.342753 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d957723d-e225-4d3d-b54f-b53ccf1a41a3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:37:38.346035 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d957723d-e225-4d3d-b54f-b53ccf1a41a3 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:37:38.385061 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 917/1821] 10.222.0.33 () {60 vars in 1152 bytes} [Mon Jul 25 05:37:38 2022] POST /identity/v3/projects => generated 385 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 25 05:37:38.387932 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-614d830c-c304-4ac8-8a58-a8d77c3df719 None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:37:38.401877 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-614d830c-c304-4ac8-8a58-a8d77c3df719 None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:37:38.435168 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:DELETE=7,keystone:SELECT=92,keystone:INSERT=1 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:37:38.522218 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d957723d-e225-4d3d-b54f-b53ccf1a41a3 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:37:38.524529 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3900d369-655c-47d4-a358-a2174c2352ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:37:38.525044 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-d957723d-e225-4d3d-b54f-b53ccf1a41a3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:37:38.528613 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3900d369-655c-47d4-a358-a2174c2352ee admin admin] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:37:38.529595 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3900d369-655c-47d4-a358-a2174c2352ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:37:38.531416 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3900d369-655c-47d4-a358-a2174c2352ee admin admin] PATH_INFO: `/v3/users` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:37:38.532018 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3900d369-655c-47d4-a358-a2174c2352ee admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:37:38.535749 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3900d369-655c-47d4-a358-a2174c2352ee admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:37:38.544754 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d957723d-e225-4d3d-b54f-b53ccf1a41a3 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:37:38.552280 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-336d5a24-d2e2-43d3-874c-5f3e8401502e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:37:38.553760 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336d5a24-d2e2-43d3-874c-5f3e8401502e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:37:38.553760 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336d5a24-d2e2-43d3-874c-5f3e8401502e admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:37:38.554216 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336d5a24-d2e2-43d3-874c-5f3e8401502e admin admin] PATH_INFO: `/v3/projects/d7a08dc773bd4cf59652c998b97752c7` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:37:38.555669 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-336d5a24-d2e2-43d3-874c-5f3e8401502e admin admin] RBAC: Authorizing `identity:delete_project(project_id=d7a08dc773bd4cf59652c998b97752c7)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:37:38.559788 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-336d5a24-d2e2-43d3-874c-5f3e8401502e admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:37:38.594549 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 905/1822] 10.222.0.33 () {60 vars in 1143 bytes} [Mon Jul 25 05:37:38 2022] POST /identity/v3/users => generated 299 bytes in 208 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 05:37:38.596219 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3900d369-655c-47d4-a358-a2174c2352ee admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:37:38.597738 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-3900d369-655c-47d4-a358-a2174c2352ee admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:37:38.617840 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3900d369-655c-47d4-a358-a2174c2352ee admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:37:38.624595 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dbbd291-8315-4816-aff2-68acd7a80b07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:37:38.625551 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dbbd291-8315-4816-aff2-68acd7a80b07 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:37:38.626333 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dbbd291-8315-4816-aff2-68acd7a80b07 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:37:38.626827 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dbbd291-8315-4816-aff2-68acd7a80b07 admin admin] PATH_INFO: `/v3/projects/c82715a2535c4d369676883b34e936e7` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:37:38.628502 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dbbd291-8315-4816-aff2-68acd7a80b07 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c82715a2535c4d369676883b34e936e7)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:37:38.631436 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dbbd291-8315-4816-aff2-68acd7a80b07 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:37:38.684031 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-336d5a24-d2e2-43d3-874c-5f3e8401502e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd7a08dc773bd4cf59652c998b97752c7', 'request_id': 'req-336d5a24-d2e2-43d3-874c-5f3e8401502e'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:37:38.690607 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-336d5a24-d2e2-43d3-874c-5f3e8401502e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd7a08dc773bd4cf59652c998b97752c7', 'request_id': 'req-336d5a24-d2e2-43d3-874c-5f3e8401502e'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:37:38.698353 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-336d5a24-d2e2-43d3-874c-5f3e8401502e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd7a08dc773bd4cf59652c998b97752c7', 'request_id': 'req-336d5a24-d2e2-43d3-874c-5f3e8401502e'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:37:38.705057 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-0dbbd291-8315-4816-aff2-68acd7a80b07 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c82715a2535c4d369676883b34e936e7', 'request_id': 'req-0dbbd291-8315-4816-aff2-68acd7a80b07'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:37:38.713430 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-0dbbd291-8315-4816-aff2-68acd7a80b07 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c82715a2535c4d369676883b34e936e7', 'request_id': 'req-0dbbd291-8315-4816-aff2-68acd7a80b07'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:37:38.716103 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-336d5a24-d2e2-43d3-874c-5f3e8401502e admin admin] The token cache is being invalidate because project d7a08dc773bd4cf59652c998b97752c7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80249) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 05:37:38.717477 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-336d5a24-d2e2-43d3-874c-5f3e8401502e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:37:38.717876 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-336d5a24-d2e2-43d3-874c-5f3e8401502e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:37:38.723207 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 918/1823] 10.222.0.33 () {58 vars in 1232 bytes} [Mon Jul 25 05:37:38 2022] DELETE /identity/v3/projects/d7a08dc773bd4cf59652c998b97752c7 => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 05:37:38.724173 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-336d5a24-d2e2-43d3-874c-5f3e8401502e admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:37:38.724301 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-0dbbd291-8315-4816-aff2-68acd7a80b07 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c82715a2535c4d369676883b34e936e7', 'request_id': 'req-0dbbd291-8315-4816-aff2-68acd7a80b07'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:37:38.725611 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-336d5a24-d2e2-43d3-874c-5f3e8401502e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:37:38.746730 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-0dbbd291-8315-4816-aff2-68acd7a80b07 admin admin] The token cache is being invalidate because project c82715a2535c4d369676883b34e936e7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80248) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 05:37:38.747107 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-0dbbd291-8315-4816-aff2-68acd7a80b07 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:37:38.747927 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-0dbbd291-8315-4816-aff2-68acd7a80b07 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:37:38.755481 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 906/1824] 10.222.0.33 () {58 vars in 1232 bytes} [Mon Jul 25 05:37:38 2022] DELETE /identity/v3/projects/c82715a2535c4d369676883b34e936e7 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 05:37:38.755481 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef097ee-0f7c-4cca-b2b9-e036e805dc8a None None] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:37:38.755481 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef097ee-0f7c-4cca-b2b9-e036e805dc8a None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:37:38.755481 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef097ee-0f7c-4cca-b2b9-e036e805dc8a None None] PATH_INFO: `/` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:37:38.755481 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 907/1825] 10.222.0.33 () {54 vars in 912 bytes} [Mon Jul 25 05:37:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:37:38.756004 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef097ee-0f7c-4cca-b2b9-e036e805dc8a None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:37:38.758046 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-0ef097ee-0f7c-4cca-b2b9-e036e805dc8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:37:38.775012 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:37:38.832935 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-336d5a24-d2e2-43d3-874c-5f3e8401502e admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:37:38.865234 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef097ee-0f7c-4cca-b2b9-e036e805dc8a None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:37:38.869856 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f187011a-a79a-423b-8b13-8f1913d8907c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:37:38.870739 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f187011a-a79a-423b-8b13-8f1913d8907c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:37:38.871025 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f187011a-a79a-423b-8b13-8f1913d8907c admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:37:38.871299 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f187011a-a79a-423b-8b13-8f1913d8907c admin admin] PATH_INFO: `/v3/users/1f3ca7d62dca4857adce7a350e388828` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:37:38.872567 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f187011a-a79a-423b-8b13-8f1913d8907c admin admin] RBAC: Authorizing `identity:delete_user(user_id=1f3ca7d62dca4857adce7a350e388828)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:37:38.874607 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f187011a-a79a-423b-8b13-8f1913d8907c admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:37:38.890033 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a00303e-1013-44a2-8551-c8e8c8d3e972 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:37:38.890707 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a00303e-1013-44a2-8551-c8e8c8d3e972 admin admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:37:38.890989 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a00303e-1013-44a2-8551-c8e8c8d3e972 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:37:38.891264 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a00303e-1013-44a2-8551-c8e8c8d3e972 admin admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:37:38.891994 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a00303e-1013-44a2-8551-c8e8c8d3e972 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:37:38.893615 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a00303e-1013-44a2-8551-c8e8c8d3e972 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:37:38.905666 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 919/1826] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jul 25 05:37:38 2022] GET /identity/v3/roles => generated 1420 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 05:37:38.906996 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a00303e-1013-44a2-8551-c8e8c8d3e972 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:37:38.910069 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-4a00303e-1013-44a2-8551-c8e8c8d3e972 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:37:38.919284 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=84,keystone:INSERT=4,keystone:DELETE=12 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:37:38.924114 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a00303e-1013-44a2-8551-c8e8c8d3e972 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:37:38.927229 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-f187011a-a79a-423b-8b13-8f1913d8907c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1f3ca7d62dca4857adce7a350e388828'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:37:38.929749 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc6af9e7-f0a1-4ce8-9fd3-9d9bc93312b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:37:38.930448 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6af9e7-f0a1-4ce8-9fd3-9d9bc93312b4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:37:38.930737 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6af9e7-f0a1-4ce8-9fd3-9d9bc93312b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:37:38.931014 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6af9e7-f0a1-4ce8-9fd3-9d9bc93312b4 admin admin] PATH_INFO: `/v3/users/e698e554d5c8486e9805f49c9671d2aa` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:37:38.932206 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc6af9e7-f0a1-4ce8-9fd3-9d9bc93312b4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e698e554d5c8486e9805f49c9671d2aa)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:37:38.934602 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc6af9e7-f0a1-4ce8-9fd3-9d9bc93312b4 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:37:38.940041 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-f187011a-a79a-423b-8b13-8f1913d8907c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1f3ca7d62dca4857adce7a350e388828'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:37:38.940733 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-f187011a-a79a-423b-8b13-8f1913d8907c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1f3ca7d62dca4857adce7a350e388828'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:37:38.948892 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-f187011a-a79a-423b-8b13-8f1913d8907c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1f3ca7d62dca4857adce7a350e388828'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:37:38.949538 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-f187011a-a79a-423b-8b13-8f1913d8907c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1f3ca7d62dca4857adce7a350e388828'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:37:38.966654 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 908/1827] 10.222.0.33 () {58 vars in 1223 bytes} [Mon Jul 25 05:37:38 2022] DELETE /identity/v3/users/1f3ca7d62dca4857adce7a350e388828 => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 05:37:38.967533 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f187011a-a79a-423b-8b13-8f1913d8907c admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:37:38.968904 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-f187011a-a79a-423b-8b13-8f1913d8907c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:37:38.998557 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-dc6af9e7-f0a1-4ce8-9fd3-9d9bc93312b4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e698e554d5c8486e9805f49c9671d2aa'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:37:38.999251 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-dc6af9e7-f0a1-4ce8-9fd3-9d9bc93312b4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e698e554d5c8486e9805f49c9671d2aa'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:37:39.009351 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-dc6af9e7-f0a1-4ce8-9fd3-9d9bc93312b4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e698e554d5c8486e9805f49c9671d2aa'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:37:39.046128 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-dc6af9e7-f0a1-4ce8-9fd3-9d9bc93312b4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e698e554d5c8486e9805f49c9671d2aa'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:37:39.046791 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f187011a-a79a-423b-8b13-8f1913d8907c admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:37:39.047719 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-dc6af9e7-f0a1-4ce8-9fd3-9d9bc93312b4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e698e554d5c8486e9805f49c9671d2aa'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:37:39.052895 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d28d9e-1568-4a40-a25f-f3c9d0d2d70f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:37:39.054475 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d28d9e-1568-4a40-a25f-f3c9d0d2d70f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:37:39.054769 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d28d9e-1568-4a40-a25f-f3c9d0d2d70f admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:37:39.055188 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d28d9e-1568-4a40-a25f-f3c9d0d2d70f admin admin] PATH_INFO: `/v3/projects/6d5fd404b69e41ac8af538fd4b4d914f` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:37:39.056678 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0d28d9e-1568-4a40-a25f-f3c9d0d2d70f admin admin] RBAC: Authorizing `identity:delete_project(project_id=6d5fd404b69e41ac8af538fd4b4d914f)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:37:39.059897 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0d28d9e-1568-4a40-a25f-f3c9d0d2d70f admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:37:39.068322 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 920/1828] 10.222.0.33 () {58 vars in 1223 bytes} [Mon Jul 25 05:37:38 2022] DELETE /identity/v3/users/e698e554d5c8486e9805f49c9671d2aa => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 05:37:39.069768 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb30f9db-7377-4136-82a8-ab3faeb2a0cb None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:37:39.070256 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb30f9db-7377-4136-82a8-ab3faeb2a0cb None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:37:39.070754 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb30f9db-7377-4136-82a8-ab3faeb2a0cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:37:39.097750 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-eb30f9db-7377-4136-82a8-ab3faeb2a0cb None None] MFA Rules not processed for user `b2931200f99e43d79a50a9f6e02738e6`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:37:39.100857 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-eb30f9db-7377-4136-82a8-ab3faeb2a0cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:37:39.152111 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-b0d28d9e-1568-4a40-a25f-f3c9d0d2d70f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6d5fd404b69e41ac8af538fd4b4d914f', 'request_id': 'req-b0d28d9e-1568-4a40-a25f-f3c9d0d2d70f'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:37:39.159044 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:37:39.159553 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-b0d28d9e-1568-4a40-a25f-f3c9d0d2d70f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6d5fd404b69e41ac8af538fd4b4d914f', 'request_id': 'req-b0d28d9e-1568-4a40-a25f-f3c9d0d2d70f'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:37:39.184639 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-b0d28d9e-1568-4a40-a25f-f3c9d0d2d70f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6d5fd404b69e41ac8af538fd4b4d914f', 'request_id': 'req-b0d28d9e-1568-4a40-a25f-f3c9d0d2d70f'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:37:39.196133 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 921/1829] 10.222.0.33 () {58 vars in 1009 bytes} [Mon Jul 25 05:37:39 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:37:39.197087 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb30f9db-7377-4136-82a8-ab3faeb2a0cb None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:37:39.199077 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-eb30f9db-7377-4136-82a8-ab3faeb2a0cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:37:39.211213 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-b0d28d9e-1568-4a40-a25f-f3c9d0d2d70f admin admin] The token cache is being invalidate because project 6d5fd404b69e41ac8af538fd4b4d914f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80248) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 05:37:39.211633 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-b0d28d9e-1568-4a40-a25f-f3c9d0d2d70f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:37:39.213133 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-b0d28d9e-1568-4a40-a25f-f3c9d0d2d70f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:37:39.217379 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 909/1830] 10.222.0.33 () {58 vars in 1232 bytes} [Mon Jul 25 05:37:38 2022] DELETE /identity/v3/projects/6d5fd404b69e41ac8af538fd4b4d914f => generated 0 bytes in 249 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 05:37:39.219793 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d28d9e-1568-4a40-a25f-f3c9d0d2d70f admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:37:39.221487 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-b0d28d9e-1568-4a40-a25f-f3c9d0d2d70f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:37:39.293985 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:DELETE=12,keystone:SELECT=86,keystone:INSERT=2 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:37:39.327221 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d28d9e-1568-4a40-a25f-f3c9d0d2d70f admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:37:39.332918 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-496e44fd-3e9e-4154-82a3-8ed9bf074757 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:37:39.333874 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496e44fd-3e9e-4154-82a3-8ed9bf074757 admin admin] REQUEST_METHOD: `PUT` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:37:39.334194 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496e44fd-3e9e-4154-82a3-8ed9bf074757 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:37:39.335411 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496e44fd-3e9e-4154-82a3-8ed9bf074757 admin admin] PATH_INFO: `/v3/projects/a15b25ea642841e3a294e3f12c500c76/users/7d6cee5b39724a68a0e8b129bb12c2bf/roles/39e5c666285a42fe8fc2c3afca6260d1` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:37:39.341834 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb30f9db-7377-4136-82a8-ab3faeb2a0cb None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:37:39.348110 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74af1a46-9534-4a05-8c0a-41dc476c80a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:37:39.349800 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74af1a46-9534-4a05-8c0a-41dc476c80a4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:37:39.350260 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74af1a46-9534-4a05-8c0a-41dc476c80a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:37:39.351032 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74af1a46-9534-4a05-8c0a-41dc476c80a4 admin admin] PATH_INFO: `/v3/projects/8b5056fa50f347ddb09bc097201ee4b9` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:37:39.352537 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74af1a46-9534-4a05-8c0a-41dc476c80a4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8b5056fa50f347ddb09bc097201ee4b9)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:37:39.352976 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-496e44fd-3e9e-4154-82a3-8ed9bf074757 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a15b25ea642841e3a294e3f12c500c76, user_id=7d6cee5b39724a68a0e8b129bb12c2bf, role_id=39e5c666285a42fe8fc2c3afca6260d1)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:37:39.356328 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74af1a46-9534-4a05-8c0a-41dc476c80a4 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:37:39.356477 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-496e44fd-3e9e-4154-82a3-8ed9bf074757 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:37:39.378424 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 910/1831] 10.222.0.33 () {60 vars in 1482 bytes} [Mon Jul 25 05:37:39 2022] PUT /identity/v3/projects/a15b25ea642841e3a294e3f12c500c76/users/7d6cee5b39724a68a0e8b129bb12c2bf/roles/39e5c666285a42fe8fc2c3afca6260d1 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 05:37:39.379413 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-496e44fd-3e9e-4154-82a3-8ed9bf074757 admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:37:39.381287 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-496e44fd-3e9e-4154-82a3-8ed9bf074757 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:37:39.453861 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-74af1a46-9534-4a05-8c0a-41dc476c80a4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8b5056fa50f347ddb09bc097201ee4b9', 'request_id': 'req-74af1a46-9534-4a05-8c0a-41dc476c80a4'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:37:39.457902 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-496e44fd-3e9e-4154-82a3-8ed9bf074757 admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:37:39.464957 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-74af1a46-9534-4a05-8c0a-41dc476c80a4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8b5056fa50f347ddb09bc097201ee4b9', 'request_id': 'req-74af1a46-9534-4a05-8c0a-41dc476c80a4'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:37:39.475910 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-74af1a46-9534-4a05-8c0a-41dc476c80a4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8b5056fa50f347ddb09bc097201ee4b9', 'request_id': 'req-74af1a46-9534-4a05-8c0a-41dc476c80a4'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:37:39.515700 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-74af1a46-9534-4a05-8c0a-41dc476c80a4 admin admin] The token cache is being invalidate because project 8b5056fa50f347ddb09bc097201ee4b9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80249) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 05:37:39.516113 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-74af1a46-9534-4a05-8c0a-41dc476c80a4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:37:39.517542 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-74af1a46-9534-4a05-8c0a-41dc476c80a4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:37:39.520385 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 922/1832] 10.222.0.33 () {58 vars in 1232 bytes} [Mon Jul 25 05:37:39 2022] DELETE /identity/v3/projects/8b5056fa50f347ddb09bc097201ee4b9 => generated 0 bytes in 324 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 05:37:39.526801 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f89452-0988-453d-9aea-76f44a9601cd None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:37:39.526801 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f89452-0988-453d-9aea-76f44a9601cd None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:37:39.527121 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f89452-0988-453d-9aea-76f44a9601cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:37:39.548814 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdfe5d11-b1c8-41c6-8f4e-72f67027a802 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b2931200f99e43d79a50a9f6e02738e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92afba266cce4d1bad4134c351e74be3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:37:39.549714 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdfe5d11-b1c8-41c6-8f4e-72f67027a802 service neutron] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:37:39.550609 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdfe5d11-b1c8-41c6-8f4e-72f67027a802 service neutron] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:37:39.550960 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdfe5d11-b1c8-41c6-8f4e-72f67027a802 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:37:39.552072 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-d9f89452-0988-453d-9aea-76f44a9601cd None None] MFA Rules not processed for user `1eae540242d94c10b0ad33960a855f08`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:37:39.553267 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-fdfe5d11-b1c8-41c6-8f4e-72f67027a802 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:37:39.553539 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-d9f89452-0988-453d-9aea-76f44a9601cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:37:39.566072 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdfe5d11-b1c8-41c6-8f4e-72f67027a802 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:37:39.567725 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdfe5d11-b1c8-41c6-8f4e-72f67027a802 service neutron] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:37:39.581866 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 911/1833] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jul 25 05:37:39 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 203 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 05:37:39.582953 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdfe5d11-b1c8-41c6-8f4e-72f67027a802 service neutron] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:37:39.585106 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-fdfe5d11-b1c8-41c6-8f4e-72f67027a802 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:37:39.640954 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 923/1834] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 25 05:37:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:37:39.641357 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f89452-0988-453d-9aea-76f44a9601cd None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:37:39.643391 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-d9f89452-0988-453d-9aea-76f44a9601cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:37:39.656201 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f89452-0988-453d-9aea-76f44a9601cd None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:37:39.661292 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1810c931-c744-4d30-a333-437b9cc94f45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:37:39.662381 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1810c931-c744-4d30-a333-437b9cc94f45 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:37:39.662767 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1810c931-c744-4d30-a333-437b9cc94f45 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:37:39.663245 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1810c931-c744-4d30-a333-437b9cc94f45 admin admin] PATH_INFO: `/v3/projects/9a4d72eaabc943c598bf30d7b744e5f1` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:37:39.664654 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1810c931-c744-4d30-a333-437b9cc94f45 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9a4d72eaabc943c598bf30d7b744e5f1)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:37:39.666721 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1810c931-c744-4d30-a333-437b9cc94f45 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:37:39.684111 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdfe5d11-b1c8-41c6-8f4e-72f67027a802 service neutron] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:37:39.688081 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:37:39.691560 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d41f1ad-cab8-4bad-9b66-8861e6224677 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:37:39.692408 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d41f1ad-cab8-4bad-9b66-8861e6224677 admin admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:37:39.693211 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d41f1ad-cab8-4bad-9b66-8861e6224677 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:37:39.693971 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d41f1ad-cab8-4bad-9b66-8861e6224677 admin admin] PATH_INFO: `/v3/roles` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:37:39.695123 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d41f1ad-cab8-4bad-9b66-8861e6224677 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:37:39.699238 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d41f1ad-cab8-4bad-9b66-8861e6224677 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:37:39.711317 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 912/1835] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jul 25 05:37:39 2022] GET /identity/v3/roles => generated 1420 bytes in 129 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 05:37:39.716346 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f294d394-f2ea-4eec-9151-85e5e7638529 None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:37:39.716647 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f294d394-f2ea-4eec-9151-85e5e7638529 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:37:39.717098 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f294d394-f2ea-4eec-9151-85e5e7638529 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:37:39.726811 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-1810c931-c744-4d30-a333-437b9cc94f45 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9a4d72eaabc943c598bf30d7b744e5f1', 'request_id': 'req-1810c931-c744-4d30-a333-437b9cc94f45'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:37:39.733567 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-1810c931-c744-4d30-a333-437b9cc94f45 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9a4d72eaabc943c598bf30d7b744e5f1', 'request_id': 'req-1810c931-c744-4d30-a333-437b9cc94f45'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:37:39.735316 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-f294d394-f2ea-4eec-9151-85e5e7638529 None None] MFA Rules not processed for user `1eae540242d94c10b0ad33960a855f08`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:37:39.736537 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-f294d394-f2ea-4eec-9151-85e5e7638529 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:37:39.741757 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-1810c931-c744-4d30-a333-437b9cc94f45 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9a4d72eaabc943c598bf30d7b744e5f1', 'request_id': 'req-1810c931-c744-4d30-a333-437b9cc94f45'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:37:39.761597 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:37:39.762763 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-1810c931-c744-4d30-a333-437b9cc94f45 admin admin] The token cache is being invalidate because project 9a4d72eaabc943c598bf30d7b744e5f1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80249) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 05:37:39.763550 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-1810c931-c744-4d30-a333-437b9cc94f45 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:37:39.764394 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-1810c931-c744-4d30-a333-437b9cc94f45 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:37:39.766268 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 924/1836] 10.222.0.33 () {58 vars in 1232 bytes} [Mon Jul 25 05:37:39 2022] DELETE /identity/v3/projects/9a4d72eaabc943c598bf30d7b744e5f1 => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 05:37:39.766936 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1810c931-c744-4d30-a333-437b9cc94f45 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:37:39.768640 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-1810c931-c744-4d30-a333-437b9cc94f45 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:37:39.821444 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 913/1837] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 25 05:37:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:37:39.824167 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141cdf7b-f9ac-488c-81e6-b4a51e0dd63a None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:37:39.824675 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141cdf7b-f9ac-488c-81e6-b4a51e0dd63a None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:37:39.825041 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141cdf7b-f9ac-488c-81e6-b4a51e0dd63a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:37:39.845819 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-141cdf7b-f9ac-488c-81e6-b4a51e0dd63a None None] MFA Rules not processed for user `1eae540242d94c10b0ad33960a855f08`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:37:39.847099 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-141cdf7b-f9ac-488c-81e6-b4a51e0dd63a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:37:39.852061 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1810c931-c744-4d30-a333-437b9cc94f45 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:37:39.855541 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 914/1838] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 25 05:37:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:37:39.857088 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-141cdf7b-f9ac-488c-81e6-b4a51e0dd63a None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:37:39.857602 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-141cdf7b-f9ac-488c-81e6-b4a51e0dd63a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:37:39.860569 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5dda199-e65b-4489-8c0a-494d68d57c36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:37:39.861696 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5dda199-e65b-4489-8c0a-494d68d57c36 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:37:39.861991 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5dda199-e65b-4489-8c0a-494d68d57c36 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:37:39.862272 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5dda199-e65b-4489-8c0a-494d68d57c36 admin admin] PATH_INFO: `/v3/projects/9555a3211d264bc5beeefbe68cccc4e1` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:37:39.863702 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5dda199-e65b-4489-8c0a-494d68d57c36 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9555a3211d264bc5beeefbe68cccc4e1)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:37:39.865762 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5dda199-e65b-4489-8c0a-494d68d57c36 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:37:39.870187 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-141cdf7b-f9ac-488c-81e6-b4a51e0dd63a None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:37:39.875348 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab03cd1-f123-479f-b6f5-1a97dd3b2915 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:37:39.876573 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab03cd1-f123-479f-b6f5-1a97dd3b2915 admin admin] REQUEST_METHOD: `PUT` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:37:39.876573 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab03cd1-f123-479f-b6f5-1a97dd3b2915 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:37:39.876956 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab03cd1-f123-479f-b6f5-1a97dd3b2915 admin admin] PATH_INFO: `/v3/projects/a15b25ea642841e3a294e3f12c500c76/users/7d6cee5b39724a68a0e8b129bb12c2bf/roles/39e5c666285a42fe8fc2c3afca6260d1` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:37:39.879107 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ab03cd1-f123-479f-b6f5-1a97dd3b2915 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a15b25ea642841e3a294e3f12c500c76, user_id=7d6cee5b39724a68a0e8b129bb12c2bf, role_id=39e5c666285a42fe8fc2c3afca6260d1)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:37:39.881898 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ab03cd1-f123-479f-b6f5-1a97dd3b2915 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:37:39.896427 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 915/1839] 10.222.0.33 () {60 vars in 1482 bytes} [Mon Jul 25 05:37:39 2022] PUT /identity/v3/projects/a15b25ea642841e3a294e3f12c500c76/users/7d6cee5b39724a68a0e8b129bb12c2bf/roles/39e5c666285a42fe8fc2c3afca6260d1 => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 05:37:39.896943 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5867569-dd6e-4f47-8a94-7216d75a4fca None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:37:39.897121 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5867569-dd6e-4f47-8a94-7216d75a4fca None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:37:39.897476 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5867569-dd6e-4f47-8a94-7216d75a4fca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:37:39.918855 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-d5867569-dd6e-4f47-8a94-7216d75a4fca None None] MFA Rules not processed for user `1eae540242d94c10b0ad33960a855f08`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:37:39.920518 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-d5867569-dd6e-4f47-8a94-7216d75a4fca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:37:39.927583 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-d5dda199-e65b-4489-8c0a-494d68d57c36 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9555a3211d264bc5beeefbe68cccc4e1', 'request_id': 'req-d5dda199-e65b-4489-8c0a-494d68d57c36'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:37:39.933596 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-d5dda199-e65b-4489-8c0a-494d68d57c36 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9555a3211d264bc5beeefbe68cccc4e1', 'request_id': 'req-d5dda199-e65b-4489-8c0a-494d68d57c36'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:37:39.940743 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-d5dda199-e65b-4489-8c0a-494d68d57c36 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9555a3211d264bc5beeefbe68cccc4e1', 'request_id': 'req-d5dda199-e65b-4489-8c0a-494d68d57c36'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:37:39.974654 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-d5dda199-e65b-4489-8c0a-494d68d57c36 admin admin] The token cache is being invalidate because project 9555a3211d264bc5beeefbe68cccc4e1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80249) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 05:37:39.977854 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-d5dda199-e65b-4489-8c0a-494d68d57c36 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:37:39.980613 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-d5dda199-e65b-4489-8c0a-494d68d57c36 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:37:39.984005 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 925/1840] 10.222.0.33 () {58 vars in 1232 bytes} [Mon Jul 25 05:37:39 2022] DELETE /identity/v3/projects/9555a3211d264bc5beeefbe68cccc4e1 => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 05:37:39.986501 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38468e2-4478-4cf7-9412-9cacc708d3d7 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:37:39.987015 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38468e2-4478-4cf7-9412-9cacc708d3d7 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:37:39.987514 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38468e2-4478-4cf7-9412-9cacc708d3d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:37:39.992759 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 916/1841] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 25 05:37:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:37:39.997922 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5867569-dd6e-4f47-8a94-7216d75a4fca None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:37:39.999503 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-d5867569-dd6e-4f47-8a94-7216d75a4fca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:37:40.006852 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-f38468e2-4478-4cf7-9412-9cacc708d3d7 None None] MFA Rules not processed for user `1eae540242d94c10b0ad33960a855f08`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:37:40.008314 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-f38468e2-4478-4cf7-9412-9cacc708d3d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:37:40.012516 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5867569-dd6e-4f47-8a94-7216d75a4fca None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:37:40.017189 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 926/1842] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 25 05:37:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:37:40.017680 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73bb1b0b-b4f0-441e-a917-1cfca5d58be0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:37:40.018547 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450f7716-4532-44c6-8ed6-d679f7f0bad4 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:37:40.018827 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73bb1b0b-b4f0-441e-a917-1cfca5d58be0 admin admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:37:40.018945 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450f7716-4532-44c6-8ed6-d679f7f0bad4 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:37:40.019237 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73bb1b0b-b4f0-441e-a917-1cfca5d58be0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:37:40.019363 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450f7716-4532-44c6-8ed6-d679f7f0bad4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:37:40.019732 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73bb1b0b-b4f0-441e-a917-1cfca5d58be0 admin admin] PATH_INFO: `/v3/domains` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:37:40.020603 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73bb1b0b-b4f0-441e-a917-1cfca5d58be0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:37:40.021015 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73bb1b0b-b4f0-441e-a917-1cfca5d58be0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:37:40.022772 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73bb1b0b-b4f0-441e-a917-1cfca5d58be0 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:37:40.049360 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-450f7716-4532-44c6-8ed6-d679f7f0bad4 None None] MFA Rules not processed for user `7d6cee5b39724a68a0e8b129bb12c2bf`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:37:40.050435 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-450f7716-4532-44c6-8ed6-d679f7f0bad4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:37:40.053330 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 917/1843] 10.222.0.33 () {58 vars in 1152 bytes} [Mon Jul 25 05:37:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:37:40.055759 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ad5674-5572-4475-9b8c-fab8294b3a7a None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:37:40.055982 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ad5674-5572-4475-9b8c-fab8294b3a7a None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:37:40.056543 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ad5674-5572-4475-9b8c-fab8294b3a7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:37:40.072046 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:37:40.087553 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-78ad5674-5572-4475-9b8c-fab8294b3a7a None None] MFA Rules not processed for user `1eae540242d94c10b0ad33960a855f08`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:37:40.089649 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-78ad5674-5572-4475-9b8c-fab8294b3a7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:37:40.099684 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 918/1844] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 25 05:37:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:37:40.100945 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c5ce06-8797-4067-bd04-6257aa0977fd None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:37:40.101284 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c5ce06-8797-4067-bd04-6257aa0977fd None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:37:40.101517 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c5ce06-8797-4067-bd04-6257aa0977fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:37:40.121056 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-81c5ce06-8797-4067-bd04-6257aa0977fd None None] MFA Rules not processed for user `1eae540242d94c10b0ad33960a855f08`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:37:40.122197 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-81c5ce06-8797-4067-bd04-6257aa0977fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:37:40.140869 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 919/1845] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 25 05:37:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:37:40.144750 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c5ce06-8797-4067-bd04-6257aa0977fd None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:37:40.148995 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 927/1846] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 25 05:37:40 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:37:40.149540 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-450f7716-4532-44c6-8ed6-d679f7f0bad4 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:37:40.157244 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c5ce06-8797-4067-bd04-6257aa0977fd None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:37:40.161217 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-450f7716-4532-44c6-8ed6-d679f7f0bad4 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:37:40.162535 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c26e4c7-7ca4-4e53-8d36-93fee5209c45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:37:40.163205 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c26e4c7-7ca4-4e53-8d36-93fee5209c45 admin admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:37:40.163493 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c26e4c7-7ca4-4e53-8d36-93fee5209c45 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:37:40.163777 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c26e4c7-7ca4-4e53-8d36-93fee5209c45 admin admin] PATH_INFO: `/v3/domains` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:37:40.164683 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c26e4c7-7ca4-4e53-8d36-93fee5209c45 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:37:40.164995 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c26e4c7-7ca4-4e53-8d36-93fee5209c45 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:37:40.166336 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa9cf049-d86a-4ec7-92d0-d9daad46ff63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:37:40.167314 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c26e4c7-7ca4-4e53-8d36-93fee5209c45 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:37:40.167693 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9cf049-d86a-4ec7-92d0-d9daad46ff63 admin admin] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:37:40.168038 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9cf049-d86a-4ec7-92d0-d9daad46ff63 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:37:40.168381 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9cf049-d86a-4ec7-92d0-d9daad46ff63 admin admin] PATH_INFO: `/v3/projects` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:37:40.169277 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa9cf049-d86a-4ec7-92d0-d9daad46ff63 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:37:40.171357 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa9cf049-d86a-4ec7-92d0-d9daad46ff63 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:37:40.185100 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 920/1847] 10.222.0.33 () {58 vars in 1152 bytes} [Mon Jul 25 05:37:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:37:40.185100 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7027831a-abb5-44ba-a065-121e158b2f24 None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:37:40.185100 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7027831a-abb5-44ba-a065-121e158b2f24 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:37:40.185100 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7027831a-abb5-44ba-a065-121e158b2f24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:37:40.202331 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 928/1848] 10.222.0.33 () {60 vars in 1152 bytes} [Mon Jul 25 05:37:40 2022] POST /identity/v3/projects => generated 387 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 05:37:40.202331 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa9cf049-d86a-4ec7-92d0-d9daad46ff63 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:37:40.207693 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-7027831a-abb5-44ba-a065-121e158b2f24 None None] MFA Rules not processed for user `1eae540242d94c10b0ad33960a855f08`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:37:40.208980 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-7027831a-abb5-44ba-a065-121e158b2f24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:37:40.274589 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa9cf049-d86a-4ec7-92d0-d9daad46ff63 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:37:40.283792 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f39c42-f879-4110-9251-0a4026c8e2f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:37:40.284478 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f39c42-f879-4110-9251-0a4026c8e2f4 admin admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:37:40.284765 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f39c42-f879-4110-9251-0a4026c8e2f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:37:40.285081 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f39c42-f879-4110-9251-0a4026c8e2f4 admin admin] PATH_INFO: `/v3/domains` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:37:40.285846 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26f39c42-f879-4110-9251-0a4026c8e2f4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:37:40.286283 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26f39c42-f879-4110-9251-0a4026c8e2f4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:37:40.288112 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26f39c42-f879-4110-9251-0a4026c8e2f4 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:37:40.299322 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 921/1849] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 25 05:37:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:37:40.300294 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7027831a-abb5-44ba-a065-121e158b2f24 None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:37:40.301874 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-7027831a-abb5-44ba-a065-121e158b2f24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:37:40.302570 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 929/1850] 10.222.0.33 () {58 vars in 1152 bytes} [Mon Jul 25 05:37:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 102 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:37:40.303569 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f39c42-f879-4110-9251-0a4026c8e2f4 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:37:40.314466 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:37:40.331129 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f39c42-f879-4110-9251-0a4026c8e2f4 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:37:40.344680 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd3ff5c-fa9c-41e0-bb3f-a80f406d37d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:37:40.348034 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd3ff5c-fa9c-41e0-bb3f-a80f406d37d1 admin admin] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:37:40.348327 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd3ff5c-fa9c-41e0-bb3f-a80f406d37d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:37:40.348613 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd3ff5c-fa9c-41e0-bb3f-a80f406d37d1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:37:40.349464 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dd3ff5c-fa9c-41e0-bb3f-a80f406d37d1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:37:40.359175 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dd3ff5c-fa9c-41e0-bb3f-a80f406d37d1 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:37:40.387281 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7027831a-abb5-44ba-a065-121e158b2f24 None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:37:40.392393 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb7ff00-1f3b-4d74-9520-a3fa17d8bf1e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db57b478c25643dbaf936f90f6e17d0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92afba266cce4d1bad4134c351e74be3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:37:40.393586 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb7ff00-1f3b-4d74-9520-a3fa17d8bf1e service cinder] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:37:40.393902 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb7ff00-1f3b-4d74-9520-a3fa17d8bf1e service cinder] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:37:40.394247 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb7ff00-1f3b-4d74-9520-a3fa17d8bf1e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:37:40.416403 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 930/1851] 10.222.0.33 () {60 vars in 1152 bytes} [Mon Jul 25 05:37:40 2022] POST /identity/v3/projects => generated 373 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 05:37:40.416403 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd3ff5c-fa9c-41e0-bb3f-a80f406d37d1 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:37:40.497408 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deb7ff00-1f3b-4d74-9520-a3fa17d8bf1e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:37:40.499375 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deb7ff00-1f3b-4d74-9520-a3fa17d8bf1e service cinder] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:37:40.513218 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 922/1852] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jul 25 05:37:40 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 213 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 05:37:40.514249 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb7ff00-1f3b-4d74-9520-a3fa17d8bf1e service cinder] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:37:40.521362 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd3ff5c-fa9c-41e0-bb3f-a80f406d37d1 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:37:40.528034 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e230f2-0ca2-452c-921f-0b61dd67eb3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:37:40.528469 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb7ff00-1f3b-4d74-9520-a3fa17d8bf1e service cinder] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:37:40.529007 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e230f2-0ca2-452c-921f-0b61dd67eb3a admin admin] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:37:40.529400 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e230f2-0ca2-452c-921f-0b61dd67eb3a admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:37:40.529776 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e230f2-0ca2-452c-921f-0b61dd67eb3a admin admin] PATH_INFO: `/v3/users` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:37:40.530881 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3e230f2-0ca2-452c-921f-0b61dd67eb3a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:37:40.533456 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7be73dd-8e10-4b5b-a6d7-0315bbd383d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:37:40.533623 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3e230f2-0ca2-452c-921f-0b61dd67eb3a admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:37:40.534141 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7be73dd-8e10-4b5b-a6d7-0315bbd383d3 admin admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:37:40.534429 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7be73dd-8e10-4b5b-a6d7-0315bbd383d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:37:40.534710 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7be73dd-8e10-4b5b-a6d7-0315bbd383d3 admin admin] PATH_INFO: `/v3/domains` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:37:40.535467 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7be73dd-8e10-4b5b-a6d7-0315bbd383d3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:37:40.536004 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7be73dd-8e10-4b5b-a6d7-0315bbd383d3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:37:40.537564 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7be73dd-8e10-4b5b-a6d7-0315bbd383d3 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:37:40.545829 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:37:40.563534 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 923/1853] 10.222.0.33 () {58 vars in 1152 bytes} [Mon Jul 25 05:37:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:37:40.564861 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7be73dd-8e10-4b5b-a6d7-0315bbd383d3 admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:37:40.566876 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 931/1854] 10.222.0.33 () {60 vars in 1143 bytes} [Mon Jul 25 05:37:40 2022] POST /identity/v3/users => generated 294 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 05:37:40.567805 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e230f2-0ca2-452c-921f-0b61dd67eb3a admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:37:40.582499 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e230f2-0ca2-452c-921f-0b61dd67eb3a admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:37:40.589704 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd2666c-853b-4e65-b1c7-428eaddc4874 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:37:40.590531 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7be73dd-8e10-4b5b-a6d7-0315bbd383d3 admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:37:40.590681 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd2666c-853b-4e65-b1c7-428eaddc4874 admin admin] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:37:40.591071 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd2666c-853b-4e65-b1c7-428eaddc4874 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:37:40.591672 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd2666c-853b-4e65-b1c7-428eaddc4874 admin admin] PATH_INFO: `/v3/users` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:37:40.592716 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dd2666c-853b-4e65-b1c7-428eaddc4874 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:37:40.595227 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dd2666c-853b-4e65-b1c7-428eaddc4874 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:37:40.595766 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b29dd93-a6fd-4708-8d8d-76d757316bd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:37:40.596315 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b29dd93-a6fd-4708-8d8d-76d757316bd4 admin admin] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:37:40.596649 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b29dd93-a6fd-4708-8d8d-76d757316bd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:37:40.597073 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b29dd93-a6fd-4708-8d8d-76d757316bd4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:37:40.597782 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b29dd93-a6fd-4708-8d8d-76d757316bd4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:37:40.599677 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b29dd93-a6fd-4708-8d8d-76d757316bd4 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:37:40.626018 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 932/1855] 10.222.0.33 () {60 vars in 1143 bytes} [Mon Jul 25 05:37:40 2022] POST /identity/v3/users => generated 287 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 05:37:40.626018 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd2666c-853b-4e65-b1c7-428eaddc4874 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:37:40.626716 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-4dd2666c-853b-4e65-b1c7-428eaddc4874 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:37:40.633757 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 924/1856] 10.222.0.33 () {60 vars in 1152 bytes} [Mon Jul 25 05:37:40 2022] POST /identity/v3/projects => generated 375 bytes in 69 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 05:37:40.634903 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b29dd93-a6fd-4708-8d8d-76d757316bd4 admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:37:40.643874 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd2666c-853b-4e65-b1c7-428eaddc4874 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:37:40.706244 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b29dd93-a6fd-4708-8d8d-76d757316bd4 admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:37:40.710828 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-243a86b4-2227-437f-a177-6c4e175ddac8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:37:40.711420 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86cd5ff8-0a67-4882-99a3-05e0967714f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:37:40.711864 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243a86b4-2227-437f-a177-6c4e175ddac8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:37:40.712092 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86cd5ff8-0a67-4882-99a3-05e0967714f6 admin admin] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:37:40.712277 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243a86b4-2227-437f-a177-6c4e175ddac8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:37:40.712539 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86cd5ff8-0a67-4882-99a3-05e0967714f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:37:40.712684 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243a86b4-2227-437f-a177-6c4e175ddac8 admin admin] PATH_INFO: `/v3/users/69dcc7bdac564a4dbba23efe6d762b58` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:37:40.712868 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86cd5ff8-0a67-4882-99a3-05e0967714f6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:37:40.713649 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86cd5ff8-0a67-4882-99a3-05e0967714f6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:37:40.714529 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-243a86b4-2227-437f-a177-6c4e175ddac8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=69dcc7bdac564a4dbba23efe6d762b58)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:37:40.716240 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86cd5ff8-0a67-4882-99a3-05e0967714f6 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:37:40.717180 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-243a86b4-2227-437f-a177-6c4e175ddac8 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:37:40.745424 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 925/1857] 10.222.0.33 () {60 vars in 1152 bytes} [Mon Jul 25 05:37:40 2022] POST /identity/v3/projects => generated 371 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 05:37:40.745424 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86cd5ff8-0a67-4882-99a3-05e0967714f6 admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:37:40.754189 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:37:40.775378 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-243a86b4-2227-437f-a177-6c4e175ddac8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '69dcc7bdac564a4dbba23efe6d762b58'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:37:40.776113 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-243a86b4-2227-437f-a177-6c4e175ddac8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '69dcc7bdac564a4dbba23efe6d762b58'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:37:40.783706 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-243a86b4-2227-437f-a177-6c4e175ddac8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '69dcc7bdac564a4dbba23efe6d762b58'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:37:40.796033 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-243a86b4-2227-437f-a177-6c4e175ddac8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '69dcc7bdac564a4dbba23efe6d762b58'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:37:40.796723 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-243a86b4-2227-437f-a177-6c4e175ddac8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '69dcc7bdac564a4dbba23efe6d762b58'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:37:40.810432 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86cd5ff8-0a67-4882-99a3-05e0967714f6 admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:37:40.815320 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e01e2974-c9ec-4983-ba90-ded3ebd49e89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:37:40.815994 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01e2974-c9ec-4983-ba90-ded3ebd49e89 admin admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:37:40.816285 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01e2974-c9ec-4983-ba90-ded3ebd49e89 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:37:40.816566 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01e2974-c9ec-4983-ba90-ded3ebd49e89 admin admin] PATH_INFO: `/v3/roles` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:37:40.817334 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e01e2974-c9ec-4983-ba90-ded3ebd49e89 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:37:40.818952 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e01e2974-c9ec-4983-ba90-ded3ebd49e89 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:37:40.819058 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 933/1858] 10.222.0.33 () {58 vars in 1223 bytes} [Mon Jul 25 05:37:40 2022] DELETE /identity/v3/users/69dcc7bdac564a4dbba23efe6d762b58 => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 05:37:40.822270 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-243a86b4-2227-437f-a177-6c4e175ddac8 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:37:40.823830 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-243a86b4-2227-437f-a177-6c4e175ddac8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:37:40.832631 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 926/1859] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jul 25 05:37:40 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 05:37:40.832631 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e01e2974-c9ec-4983-ba90-ded3ebd49e89 admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:37:40.834899 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-e01e2974-c9ec-4983-ba90-ded3ebd49e89 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:37:40.848472 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e01e2974-c9ec-4983-ba90-ded3ebd49e89 admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:37:40.883727 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:INSERT=5,keystone:SELECT=85,keystone:DELETE=10 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:37:40.892148 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-243a86b4-2227-437f-a177-6c4e175ddac8 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:37:40.899930 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d74714-b6b8-4434-80cf-3744fb62255a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:37:40.900606 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d74714-b6b8-4434-80cf-3744fb62255a admin admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:37:40.901006 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d74714-b6b8-4434-80cf-3744fb62255a admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:37:40.902080 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d74714-b6b8-4434-80cf-3744fb62255a admin admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:37:40.902893 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55d74714-b6b8-4434-80cf-3744fb62255a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:37:40.904497 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55d74714-b6b8-4434-80cf-3744fb62255a admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:37:40.912961 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2458efa8-152a-41a5-95fb-1e049ebc4f58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:37:40.913645 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2458efa8-152a-41a5-95fb-1e049ebc4f58 admin admin] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:37:40.913938 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2458efa8-152a-41a5-95fb-1e049ebc4f58 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:37:40.914221 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2458efa8-152a-41a5-95fb-1e049ebc4f58 admin admin] PATH_INFO: `/v3/users` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:37:40.914928 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2458efa8-152a-41a5-95fb-1e049ebc4f58 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:37:40.915457 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 934/1860] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jul 25 05:37:40 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 05:37:40.916341 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d74714-b6b8-4434-80cf-3744fb62255a admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:37:40.916925 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2458efa8-152a-41a5-95fb-1e049ebc4f58 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:37:40.919706 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-55d74714-b6b8-4434-80cf-3744fb62255a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:37:40.935567 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d74714-b6b8-4434-80cf-3744fb62255a admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:37:40.938215 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 927/1861] 10.222.0.33 () {60 vars in 1143 bytes} [Mon Jul 25 05:37:40 2022] POST /identity/v3/users => generated 288 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 05:37:40.939613 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6a5a53-fb53-4758-b220-bc02a70dd387 None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:37:40.939906 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6a5a53-fb53-4758-b220-bc02a70dd387 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:37:40.940253 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6a5a53-fb53-4758-b220-bc02a70dd387 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:37:40.943138 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c34844-e4d8-44c8-959a-d17a194ef537 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:37:40.943871 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c34844-e4d8-44c8-959a-d17a194ef537 admin admin] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:37:40.944162 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c34844-e4d8-44c8-959a-d17a194ef537 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:37:40.944441 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c34844-e4d8-44c8-959a-d17a194ef537 admin admin] PATH_INFO: `/v3/users` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:37:40.945174 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66c34844-e4d8-44c8-959a-d17a194ef537 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:37:40.947083 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66c34844-e4d8-44c8-959a-d17a194ef537 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:37:40.960648 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-6f6a5a53-fb53-4758-b220-bc02a70dd387 None None] MFA Rules not processed for user `1eae540242d94c10b0ad33960a855f08`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:37:40.962127 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-6f6a5a53-fb53-4758-b220-bc02a70dd387 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:37:40.970314 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 935/1862] 10.222.0.33 () {60 vars in 1143 bytes} [Mon Jul 25 05:37:40 2022] POST /identity/v3/users => generated 286 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 05:37:40.970749 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 928/1863] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 25 05:37:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:37:40.974097 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c34844-e4d8-44c8-959a-d17a194ef537 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:37:40.974358 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f6a5a53-fb53-4758-b220-bc02a70dd387 None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:37:40.975863 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-66c34844-e4d8-44c8-959a-d17a194ef537 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:37:40.984602 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f6a5a53-fb53-4758-b220-bc02a70dd387 None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:37:40.987850 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c34844-e4d8-44c8-959a-d17a194ef537 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:37:40.989248 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a3cb6a-9e32-4cfe-9802-dd52d6dbfdf6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:37:40.990022 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a3cb6a-9e32-4cfe-9802-dd52d6dbfdf6 admin admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:37:40.990313 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a3cb6a-9e32-4cfe-9802-dd52d6dbfdf6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:37:40.990598 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a3cb6a-9e32-4cfe-9802-dd52d6dbfdf6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:37:40.991343 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5a3cb6a-9e32-4cfe-9802-dd52d6dbfdf6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:37:40.992328 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aaac95a-29d1-495d-a141-3fd3e01528df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:37:40.992973 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5a3cb6a-9e32-4cfe-9802-dd52d6dbfdf6 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:37:40.993091 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aaac95a-29d1-495d-a141-3fd3e01528df admin admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:37:40.993281 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aaac95a-29d1-495d-a141-3fd3e01528df admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:37:40.993560 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aaac95a-29d1-495d-a141-3fd3e01528df admin admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:37:40.994289 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aaac95a-29d1-495d-a141-3fd3e01528df admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:37:40.996583 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aaac95a-29d1-495d-a141-3fd3e01528df admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:37:41.003777 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 929/1864] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jul 25 05:37:40 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 05:37:41.009723 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 936/1865] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jul 25 05:37:40 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 05:37:41.009723 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aaac95a-29d1-495d-a141-3fd3e01528df admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:37:41.009920 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a3cb6a-9e32-4cfe-9802-dd52d6dbfdf6 admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:37:41.018338 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aaac95a-29d1-495d-a141-3fd3e01528df admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:37:41.019802 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a3cb6a-9e32-4cfe-9802-dd52d6dbfdf6 admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:37:41.023134 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d14baf-721a-4b65-a8f6-d89984b043eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:37:41.023799 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d14baf-721a-4b65-a8f6-d89984b043eb admin admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:37:41.024096 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d14baf-721a-4b65-a8f6-d89984b043eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:37:41.024380 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d14baf-721a-4b65-a8f6-d89984b043eb admin admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:37:41.025029 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a01bdf9e-9390-4144-bb7d-fed2371d8fe8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:37:41.025148 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57d14baf-721a-4b65-a8f6-d89984b043eb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:37:41.025820 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01bdf9e-9390-4144-bb7d-fed2371d8fe8 admin admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:37:41.026744 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01bdf9e-9390-4144-bb7d-fed2371d8fe8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:37:41.027083 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01bdf9e-9390-4144-bb7d-fed2371d8fe8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:37:41.027642 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57d14baf-721a-4b65-a8f6-d89984b043eb admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:37:41.027930 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a01bdf9e-9390-4144-bb7d-fed2371d8fe8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:37:41.029587 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a01bdf9e-9390-4144-bb7d-fed2371d8fe8 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:37:41.038902 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 937/1866] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jul 25 05:37:41 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 05:37:41.039970 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 930/1867] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jul 25 05:37:41 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 05:37:41.042618 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a01bdf9e-9390-4144-bb7d-fed2371d8fe8 admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:37:41.043031 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d14baf-721a-4b65-a8f6-d89984b043eb admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:37:41.044734 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-57d14baf-721a-4b65-a8f6-d89984b043eb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:37:41.052899 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a01bdf9e-9390-4144-bb7d-fed2371d8fe8 admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:37:41.058915 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5740427c-b6da-4534-af96-daa3ef715d5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:37:41.059819 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5740427c-b6da-4534-af96-daa3ef715d5b admin admin] REQUEST_METHOD: `PUT` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:37:41.060223 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5740427c-b6da-4534-af96-daa3ef715d5b admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:37:41.060494 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5740427c-b6da-4534-af96-daa3ef715d5b admin admin] PATH_INFO: `/v3/projects/7dc9e81bf3f44ebca324dad8bb89b55e/users/a42f1441b8564b5a82c79798deb910d5/roles/5074e74e35f54d4b9bb415c34999a82a` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:37:41.075099 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5740427c-b6da-4534-af96-daa3ef715d5b admin admin] RBAC: Authorizing `identity:create_grant(project_id=7dc9e81bf3f44ebca324dad8bb89b55e, user_id=a42f1441b8564b5a82c79798deb910d5, role_id=5074e74e35f54d4b9bb415c34999a82a)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:37:41.078318 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5740427c-b6da-4534-af96-daa3ef715d5b admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:37:41.099205 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 931/1868] 10.222.0.33 () {60 vars in 1482 bytes} [Mon Jul 25 05:37:41 2022] PUT /identity/v3/projects/7dc9e81bf3f44ebca324dad8bb89b55e/users/a42f1441b8564b5a82c79798deb910d5/roles/5074e74e35f54d4b9bb415c34999a82a => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 05:37:41.100024 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5740427c-b6da-4534-af96-daa3ef715d5b admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:37:41.111013 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d14baf-721a-4b65-a8f6-d89984b043eb admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:37:41.153204 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:37:41.170035 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-558cdaa5-5f60-4af3-954b-9f70cd76a4d6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b2931200f99e43d79a50a9f6e02738e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92afba266cce4d1bad4134c351e74be3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:37:41.170625 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558cdaa5-5f60-4af3-954b-9f70cd76a4d6 service neutron] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:37:41.170979 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558cdaa5-5f60-4af3-954b-9f70cd76a4d6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:37:41.171480 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558cdaa5-5f60-4af3-954b-9f70cd76a4d6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:37:41.179290 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5740427c-b6da-4534-af96-daa3ef715d5b admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:37:41.185139 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bebccf53-fe26-4025-8289-353fd6baafe3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:37:41.186037 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bebccf53-fe26-4025-8289-353fd6baafe3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:37:41.186326 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bebccf53-fe26-4025-8289-353fd6baafe3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:37:41.186604 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bebccf53-fe26-4025-8289-353fd6baafe3 admin admin] PATH_INFO: `/v3/projects/90862fc01a7b48e38ba4ef60673c253e/users/0d6817b3321748b59a8006cc96b63633/roles/5074e74e35f54d4b9bb415c34999a82a` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:37:41.189249 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-558cdaa5-5f60-4af3-954b-9f70cd76a4d6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:37:41.190710 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-558cdaa5-5f60-4af3-954b-9f70cd76a4d6 service neutron] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:37:41.201213 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 938/1869] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jul 25 05:37:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 162 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 05:37:41.201450 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bebccf53-fe26-4025-8289-353fd6baafe3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=90862fc01a7b48e38ba4ef60673c253e, user_id=0d6817b3321748b59a8006cc96b63633, role_id=5074e74e35f54d4b9bb415c34999a82a)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:37:41.202035 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-558cdaa5-5f60-4af3-954b-9f70cd76a4d6 service neutron] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:37:41.204424 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bebccf53-fe26-4025-8289-353fd6baafe3 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:37:41.211892 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-558cdaa5-5f60-4af3-954b-9f70cd76a4d6 service neutron] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:37:41.221076 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aed15e0-8e15-466c-8cea-64e78324371e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:37:41.221910 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aed15e0-8e15-466c-8cea-64e78324371e admin admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:37:41.222219 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aed15e0-8e15-466c-8cea-64e78324371e admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:37:41.222326 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 932/1870] 10.222.0.33 () {60 vars in 1482 bytes} [Mon Jul 25 05:37:41 2022] PUT /identity/v3/projects/90862fc01a7b48e38ba4ef60673c253e/users/0d6817b3321748b59a8006cc96b63633/roles/5074e74e35f54d4b9bb415c34999a82a => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 05:37:41.222486 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aed15e0-8e15-466c-8cea-64e78324371e admin admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:37:41.223062 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bebccf53-fe26-4025-8289-353fd6baafe3 admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:37:41.223286 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aed15e0-8e15-466c-8cea-64e78324371e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:37:41.224931 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aed15e0-8e15-466c-8cea-64e78324371e admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:37:41.240278 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 939/1871] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jul 25 05:37:41 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 05:37:41.242379 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e8e337-17ac-46e0-b792-73ca67afb1dc None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:37:41.243374 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e8e337-17ac-46e0-b792-73ca67afb1dc None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:37:41.243788 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e8e337-17ac-46e0-b792-73ca67afb1dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:37:41.274457 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-e5e8e337-17ac-46e0-b792-73ca67afb1dc None None] MFA Rules not processed for user `a42f1441b8564b5a82c79798deb910d5`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:37:41.275637 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-e5e8e337-17ac-46e0-b792-73ca67afb1dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:37:41.318128 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bebccf53-fe26-4025-8289-353fd6baafe3 admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:37:41.331516 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:37:41.332120 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12928218-35e0-4970-9406-04169c483ff0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:37:41.333137 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12928218-35e0-4970-9406-04169c483ff0 admin admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:37:41.333761 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12928218-35e0-4970-9406-04169c483ff0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:37:41.333761 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12928218-35e0-4970-9406-04169c483ff0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:37:41.337473 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12928218-35e0-4970-9406-04169c483ff0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:37:41.343667 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12928218-35e0-4970-9406-04169c483ff0 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:37:41.353995 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 940/1872] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 25 05:37:41 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:37:41.355286 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e501170-f1c0-4ab3-8d20-5be90e1e3ed4 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:37:41.355578 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e501170-f1c0-4ab3-8d20-5be90e1e3ed4 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:37:41.355864 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e501170-f1c0-4ab3-8d20-5be90e1e3ed4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:37:41.371079 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 933/1873] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jul 25 05:37:41 2022] GET /identity/v3/roles => generated 1420 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 05:37:41.371904 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12928218-35e0-4970-9406-04169c483ff0 admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:37:41.392610 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12928218-35e0-4970-9406-04169c483ff0 admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:37:41.405701 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82882b03-a724-4b2c-a7db-c2cfac3b514b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:37:41.406830 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-7e501170-f1c0-4ab3-8d20-5be90e1e3ed4 None None] MFA Rules not processed for user `0d6817b3321748b59a8006cc96b63633`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:37:41.407851 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82882b03-a724-4b2c-a7db-c2cfac3b514b admin admin] REQUEST_METHOD: `PUT` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:37:41.408804 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82882b03-a724-4b2c-a7db-c2cfac3b514b admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:37:41.409540 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82882b03-a724-4b2c-a7db-c2cfac3b514b admin admin] PATH_INFO: `/v3/projects/b9b8418e171a4f92b5b99879dcfc6581/users/be5eb59db01b48448fef59467ee778ca/roles/5074e74e35f54d4b9bb415c34999a82a` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:37:41.410735 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-7e501170-f1c0-4ab3-8d20-5be90e1e3ed4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:37:41.443578 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82882b03-a724-4b2c-a7db-c2cfac3b514b admin admin] RBAC: Authorizing `identity:create_grant(project_id=b9b8418e171a4f92b5b99879dcfc6581, user_id=be5eb59db01b48448fef59467ee778ca, role_id=5074e74e35f54d4b9bb415c34999a82a)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:37:41.450930 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82882b03-a724-4b2c-a7db-c2cfac3b514b admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:37:41.477624 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 934/1874] 10.222.0.33 () {60 vars in 1482 bytes} [Mon Jul 25 05:37:41 2022] PUT /identity/v3/projects/b9b8418e171a4f92b5b99879dcfc6581/users/be5eb59db01b48448fef59467ee778ca/roles/5074e74e35f54d4b9bb415c34999a82a => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 05:37:41.479865 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82882b03-a724-4b2c-a7db-c2cfac3b514b admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:37:41.525446 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 941/1875] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 25 05:37:41 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:37:41.525446 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e501170-f1c0-4ab3-8d20-5be90e1e3ed4 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:37:41.539083 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e501170-f1c0-4ab3-8d20-5be90e1e3ed4 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:37:41.600938 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82882b03-a724-4b2c-a7db-c2cfac3b514b admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:37:41.608689 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c382843-9e4b-4913-9ffe-d1e74a46e815 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:37:41.610302 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c382843-9e4b-4913-9ffe-d1e74a46e815 admin admin] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:37:41.610781 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c382843-9e4b-4913-9ffe-d1e74a46e815 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:37:41.611842 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c382843-9e4b-4913-9ffe-d1e74a46e815 admin admin] PATH_INFO: `/v3/projects` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:37:41.613681 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c382843-9e4b-4913-9ffe-d1e74a46e815 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:37:41.616606 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c382843-9e4b-4913-9ffe-d1e74a46e815 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:37:41.643065 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f223cf29-1915-421d-887d-bc9af8017232 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:37:41.644232 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f223cf29-1915-421d-887d-bc9af8017232 admin admin] REQUEST_METHOD: `PUT` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:37:41.645035 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f223cf29-1915-421d-887d-bc9af8017232 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:37:41.645035 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f223cf29-1915-421d-887d-bc9af8017232 admin admin] PATH_INFO: `/v3/projects/d458ef950ad24306a2be36a2aafd0ad5/users/124f328eaeaf498dbd7258524dabd801/roles/5074e74e35f54d4b9bb415c34999a82a` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:37:41.661908 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 935/1876] 10.222.0.33 () {60 vars in 1152 bytes} [Mon Jul 25 05:37:41 2022] POST /identity/v3/projects => generated 373 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 05:37:41.663157 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc8a7f9-15d1-4933-a256-dbb9a693c08b None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:37:41.663623 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc8a7f9-15d1-4933-a256-dbb9a693c08b None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:37:41.664066 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc8a7f9-15d1-4933-a256-dbb9a693c08b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:37:41.679499 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f223cf29-1915-421d-887d-bc9af8017232 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d458ef950ad24306a2be36a2aafd0ad5, user_id=124f328eaeaf498dbd7258524dabd801, role_id=5074e74e35f54d4b9bb415c34999a82a)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:37:41.682882 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f223cf29-1915-421d-887d-bc9af8017232 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:37:41.695407 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-9dc8a7f9-15d1-4933-a256-dbb9a693c08b None None] MFA Rules not processed for user `1eae540242d94c10b0ad33960a855f08`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:37:41.697234 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-9dc8a7f9-15d1-4933-a256-dbb9a693c08b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:37:41.714776 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:37:41.719977 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 942/1877] 10.222.0.33 () {60 vars in 1482 bytes} [Mon Jul 25 05:37:41 2022] PUT /identity/v3/projects/d458ef950ad24306a2be36a2aafd0ad5/users/124f328eaeaf498dbd7258524dabd801/roles/5074e74e35f54d4b9bb415c34999a82a => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 05:37:41.721176 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f223cf29-1915-421d-887d-bc9af8017232 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:37:41.834850 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 936/1878] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 25 05:37:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:37:41.835843 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc8a7f9-15d1-4933-a256-dbb9a693c08b None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:37:41.844648 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f223cf29-1915-421d-887d-bc9af8017232 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:37:41.854451 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-019bae76-320e-4683-9561-0abee9ec180d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:37:41.855283 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019bae76-320e-4683-9561-0abee9ec180d admin admin] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:37:41.857477 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019bae76-320e-4683-9561-0abee9ec180d admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:37:41.857874 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019bae76-320e-4683-9561-0abee9ec180d admin admin] PATH_INFO: `/v3/projects` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:37:41.858696 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-019bae76-320e-4683-9561-0abee9ec180d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:37:41.861716 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc8a7f9-15d1-4933-a256-dbb9a693c08b None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:37:41.862487 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-019bae76-320e-4683-9561-0abee9ec180d admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:37:41.874700 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d7336b-702c-49c3-ade5-bea8c4863738 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:37:41.875393 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d7336b-702c-49c3-ade5-bea8c4863738 admin admin] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:37:41.875680 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d7336b-702c-49c3-ade5-bea8c4863738 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:37:41.875956 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d7336b-702c-49c3-ade5-bea8c4863738 admin admin] PATH_INFO: `/v3/users` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:37:41.876408 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:37:41.876658 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4d7336b-702c-49c3-ade5-bea8c4863738 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:37:41.886834 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4d7336b-702c-49c3-ade5-bea8c4863738 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:37:41.925726 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 937/1879] 10.222.0.33 () {60 vars in 1143 bytes} [Mon Jul 25 05:37:41 2022] POST /identity/v3/users => generated 293 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 05:37:41.927598 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8a5e9c-4e73-4ffb-bb0a-5c4bcc67ddbd None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:37:41.928032 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8a5e9c-4e73-4ffb-bb0a-5c4bcc67ddbd None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:37:41.928439 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8a5e9c-4e73-4ffb-bb0a-5c4bcc67ddbd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:37:41.946957 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 943/1880] 10.222.0.33 () {60 vars in 1152 bytes} [Mon Jul 25 05:37:41 2022] POST /identity/v3/projects => generated 385 bytes in 226 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 05:37:41.947916 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-019bae76-320e-4683-9561-0abee9ec180d admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:37:41.968179 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-4c8a5e9c-4e73-4ffb-bb0a-5c4bcc67ddbd None None] MFA Rules not processed for user `124f328eaeaf498dbd7258524dabd801`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:37:41.969656 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-4c8a5e9c-4e73-4ffb-bb0a-5c4bcc67ddbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:37:42.019917 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-019bae76-320e-4683-9561-0abee9ec180d admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:37:42.025096 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d0d5757-b99d-47ad-ba91-5fb650b97645 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b2931200f99e43d79a50a9f6e02738e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92afba266cce4d1bad4134c351e74be3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:37:42.025873 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0d5757-b99d-47ad-ba91-5fb650b97645 service neutron] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:37:42.027335 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0d5757-b99d-47ad-ba91-5fb650b97645 service neutron] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:37:42.027725 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0d5757-b99d-47ad-ba91-5fb650b97645 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:37:42.068736 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 938/1881] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 25 05:37:41 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:37:42.069543 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c8a5e9c-4e73-4ffb-bb0a-5c4bcc67ddbd None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:37:42.071292 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-4c8a5e9c-4e73-4ffb-bb0a-5c4bcc67ddbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:37:42.123624 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d0d5757-b99d-47ad-ba91-5fb650b97645 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:37:42.125715 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d0d5757-b99d-47ad-ba91-5fb650b97645 service neutron] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:37:42.138708 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 944/1882] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jul 25 05:37:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 191 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 05:37:42.139679 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d0d5757-b99d-47ad-ba91-5fb650b97645 service neutron] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:37:42.153677 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d0d5757-b99d-47ad-ba91-5fb650b97645 service neutron] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:37:42.159958 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b08cdc78-445f-4aeb-a55c-e3dfa3e3aee0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:37:42.161762 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08cdc78-445f-4aeb-a55c-e3dfa3e3aee0 admin admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:37:42.162157 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08cdc78-445f-4aeb-a55c-e3dfa3e3aee0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:37:42.162627 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08cdc78-445f-4aeb-a55c-e3dfa3e3aee0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:37:42.163438 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b08cdc78-445f-4aeb-a55c-e3dfa3e3aee0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:37:42.166354 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b08cdc78-445f-4aeb-a55c-e3dfa3e3aee0 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:37:42.185762 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 945/1883] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jul 25 05:37:42 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 05:37:42.187137 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b08cdc78-445f-4aeb-a55c-e3dfa3e3aee0 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:37:42.219405 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b08cdc78-445f-4aeb-a55c-e3dfa3e3aee0 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:37:42.232684 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e89daa-2acc-4d68-9f45-247fb06cc4ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:37:42.234347 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e89daa-2acc-4d68-9f45-247fb06cc4ad admin admin] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:37:42.234763 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e89daa-2acc-4d68-9f45-247fb06cc4ad admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:37:42.235053 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e89daa-2acc-4d68-9f45-247fb06cc4ad admin admin] PATH_INFO: `/v3/users` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:37:42.235764 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0e89daa-2acc-4d68-9f45-247fb06cc4ad admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:37:42.240249 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c8a5e9c-4e73-4ffb-bb0a-5c4bcc67ddbd None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:37:42.242002 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0e89daa-2acc-4d68-9f45-247fb06cc4ad admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:37:42.265208 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9fdbb64-875d-4689-9667-1b577ddd2ed6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:37:42.266110 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9fdbb64-875d-4689-9667-1b577ddd2ed6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:37:42.266405 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9fdbb64-875d-4689-9667-1b577ddd2ed6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:37:42.266690 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9fdbb64-875d-4689-9667-1b577ddd2ed6 admin admin] PATH_INFO: `/v3/projects/8dc782bcea24493e96f22f968c2b8e6c` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:37:42.268936 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9fdbb64-875d-4689-9667-1b577ddd2ed6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8dc782bcea24493e96f22f968c2b8e6c)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:37:42.274088 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9fdbb64-875d-4689-9667-1b577ddd2ed6 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:37:42.294186 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 946/1884] 10.222.0.33 () {60 vars in 1143 bytes} [Mon Jul 25 05:37:42 2022] POST /identity/v3/users => generated 299 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 05:37:42.294186 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e89daa-2acc-4d68-9f45-247fb06cc4ad admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:37:42.315427 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e89daa-2acc-4d68-9f45-247fb06cc4ad admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:37:42.329840 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e544ac-d2b8-43d1-a705-e4c9cce2c47b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:37:42.330689 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e544ac-d2b8-43d1-a705-e4c9cce2c47b admin admin] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:37:42.330987 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e544ac-d2b8-43d1-a705-e4c9cce2c47b admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:37:42.331274 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e544ac-d2b8-43d1-a705-e4c9cce2c47b admin admin] PATH_INFO: `/v3/projects` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:37:42.331999 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4e544ac-d2b8-43d1-a705-e4c9cce2c47b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:37:42.333980 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4e544ac-d2b8-43d1-a705-e4c9cce2c47b admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:37:42.378656 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=95,keystone:INSERT=3,keystone:DELETE=2 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:37:42.398731 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 947/1885] 10.222.0.33 () {60 vars in 1152 bytes} [Mon Jul 25 05:37:42 2022] POST /identity/v3/projects => generated 377 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 05:37:42.408479 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e544ac-d2b8-43d1-a705-e4c9cce2c47b admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:37:42.410520 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-b9fdbb64-875d-4689-9667-1b577ddd2ed6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8dc782bcea24493e96f22f968c2b8e6c', 'request_id': 'req-b9fdbb64-875d-4689-9667-1b577ddd2ed6'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:37:42.426129 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-b9fdbb64-875d-4689-9667-1b577ddd2ed6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8dc782bcea24493e96f22f968c2b8e6c', 'request_id': 'req-b9fdbb64-875d-4689-9667-1b577ddd2ed6'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:37:42.435980 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-b9fdbb64-875d-4689-9667-1b577ddd2ed6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8dc782bcea24493e96f22f968c2b8e6c', 'request_id': 'req-b9fdbb64-875d-4689-9667-1b577ddd2ed6'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:37:42.455056 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-b9fdbb64-875d-4689-9667-1b577ddd2ed6 admin admin] The token cache is being invalidate because project 8dc782bcea24493e96f22f968c2b8e6c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80248) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 05:37:42.455682 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-b9fdbb64-875d-4689-9667-1b577ddd2ed6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:37:42.456506 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-b9fdbb64-875d-4689-9667-1b577ddd2ed6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:37:42.458700 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 939/1886] 10.222.0.33 () {58 vars in 1232 bytes} [Mon Jul 25 05:37:42 2022] DELETE /identity/v3/projects/8dc782bcea24493e96f22f968c2b8e6c => generated 0 bytes in 389 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 05:37:42.459410 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9fdbb64-875d-4689-9667-1b577ddd2ed6 admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:37:42.461756 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-b9fdbb64-875d-4689-9667-1b577ddd2ed6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:37:42.491539 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:37:42.516200 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e544ac-d2b8-43d1-a705-e4c9cce2c47b admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:37:42.553702 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf86929-970f-428e-bba0-64e0562d33cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:37:42.554659 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf86929-970f-428e-bba0-64e0562d33cd admin admin] REQUEST_METHOD: `PUT` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:37:42.554946 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf86929-970f-428e-bba0-64e0562d33cd admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:37:42.555227 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf86929-970f-428e-bba0-64e0562d33cd admin admin] PATH_INFO: `/v3/projects/d799bb4c8a1d45f8874976be48c31b3a/users/3e3a518b2cf54bf2b7dc8e84295782c2/roles/39e5c666285a42fe8fc2c3afca6260d1` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:37:42.580467 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdf86929-970f-428e-bba0-64e0562d33cd admin admin] RBAC: Authorizing `identity:create_grant(project_id=d799bb4c8a1d45f8874976be48c31b3a, user_id=3e3a518b2cf54bf2b7dc8e84295782c2, role_id=39e5c666285a42fe8fc2c3afca6260d1)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:37:42.583251 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdf86929-970f-428e-bba0-64e0562d33cd admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:37:42.595213 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9fdbb64-875d-4689-9667-1b577ddd2ed6 admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:37:42.606546 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 948/1887] 10.222.0.33 () {60 vars in 1482 bytes} [Mon Jul 25 05:37:42 2022] PUT /identity/v3/projects/d799bb4c8a1d45f8874976be48c31b3a/users/3e3a518b2cf54bf2b7dc8e84295782c2/roles/39e5c666285a42fe8fc2c3afca6260d1 => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 05:37:42.607554 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf86929-970f-428e-bba0-64e0562d33cd admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:37:42.610036 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-cdf86929-970f-428e-bba0-64e0562d33cd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:37:42.695260 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b86ae1-077f-46e6-b33c-255d56ce584e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:37:42.695904 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b86ae1-077f-46e6-b33c-255d56ce584e admin admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:37:42.696198 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b86ae1-077f-46e6-b33c-255d56ce584e admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:37:42.696505 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b86ae1-077f-46e6-b33c-255d56ce584e admin admin] PATH_INFO: `/v3/roles` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:37:42.697469 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4b86ae1-077f-46e6-b33c-255d56ce584e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:37:42.704010 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4b86ae1-077f-46e6-b33c-255d56ce584e admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:37:42.705241 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf86929-970f-428e-bba0-64e0562d33cd admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:37:42.712727 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-246bca5c-122a-4619-beba-29eeddd78b52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:37:42.713805 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246bca5c-122a-4619-beba-29eeddd78b52 admin admin] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:37:42.716147 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246bca5c-122a-4619-beba-29eeddd78b52 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:37:42.716495 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 940/1888] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jul 25 05:37:42 2022] GET /identity/v3/roles => generated 1420 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 05:37:42.717275 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246bca5c-122a-4619-beba-29eeddd78b52 admin admin] PATH_INFO: `/v3/users` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:37:42.718288 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-246bca5c-122a-4619-beba-29eeddd78b52 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:37:42.720544 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b86ae1-077f-46e6-b33c-255d56ce584e admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:37:42.722816 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-b4b86ae1-077f-46e6-b33c-255d56ce584e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:37:42.732407 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-246bca5c-122a-4619-beba-29eeddd78b52 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:37:42.736876 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b86ae1-077f-46e6-b33c-255d56ce584e admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:37:42.743874 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cbd567d-351e-436d-beec-8f1452136fe3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:37:42.745392 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cbd567d-351e-436d-beec-8f1452136fe3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:37:42.746883 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cbd567d-351e-436d-beec-8f1452136fe3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:37:42.746883 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cbd567d-351e-436d-beec-8f1452136fe3 admin admin] PATH_INFO: `/v3/users/7d6cee5b39724a68a0e8b129bb12c2bf` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:37:42.747423 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cbd567d-351e-436d-beec-8f1452136fe3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7d6cee5b39724a68a0e8b129bb12c2bf)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:37:42.750229 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cbd567d-351e-436d-beec-8f1452136fe3 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:37:42.839832 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 949/1889] 10.222.0.33 () {60 vars in 1143 bytes} [Mon Jul 25 05:37:42 2022] POST /identity/v3/users => generated 289 bytes in 232 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 05:37:42.842895 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-246bca5c-122a-4619-beba-29eeddd78b52 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:37:42.847272 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-246bca5c-122a-4619-beba-29eeddd78b52 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:37:42.875098 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-246bca5c-122a-4619-beba-29eeddd78b52 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:37:42.885272 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8890419d-9502-4174-a293-4f03826eec1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:37:42.887988 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8890419d-9502-4174-a293-4f03826eec1e admin admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:37:42.888495 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8890419d-9502-4174-a293-4f03826eec1e admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:37:42.889003 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8890419d-9502-4174-a293-4f03826eec1e admin admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:37:42.890272 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8890419d-9502-4174-a293-4f03826eec1e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:37:42.892271 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8890419d-9502-4174-a293-4f03826eec1e admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:37:42.902216 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-1cbd567d-351e-436d-beec-8f1452136fe3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7d6cee5b39724a68a0e8b129bb12c2bf'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:37:42.918855 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:DELETE=13,keystone:SELECT=86,keystone:INSERT=1 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:37:42.921690 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-1cbd567d-351e-436d-beec-8f1452136fe3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7d6cee5b39724a68a0e8b129bb12c2bf'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:37:42.922511 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-1cbd567d-351e-436d-beec-8f1452136fe3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7d6cee5b39724a68a0e8b129bb12c2bf'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:37:42.938483 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-1cbd567d-351e-436d-beec-8f1452136fe3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7d6cee5b39724a68a0e8b129bb12c2bf'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:37:42.939030 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 950/1890] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jul 25 05:37:42 2022] GET /identity/v3/roles => generated 1420 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 05:37:42.939203 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-1cbd567d-351e-436d-beec-8f1452136fe3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7d6cee5b39724a68a0e8b129bb12c2bf'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:37:42.942612 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8890419d-9502-4174-a293-4f03826eec1e admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:37:42.947664 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-8890419d-9502-4174-a293-4f03826eec1e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:37:42.965908 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 941/1891] 10.222.0.33 () {58 vars in 1223 bytes} [Mon Jul 25 05:37:42 2022] DELETE /identity/v3/users/7d6cee5b39724a68a0e8b129bb12c2bf => generated 0 bytes in 249 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 05:37:42.967886 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cbd567d-351e-436d-beec-8f1452136fe3 admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:37:42.973087 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-1cbd567d-351e-436d-beec-8f1452136fe3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:37:42.973952 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8890419d-9502-4174-a293-4f03826eec1e admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:37:43.091316 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddee8e88-bf60-436d-a898-87aa5f35838e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:37:43.092172 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddee8e88-bf60-436d-a898-87aa5f35838e admin admin] REQUEST_METHOD: `PUT` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:37:43.092464 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddee8e88-bf60-436d-a898-87aa5f35838e admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:37:43.092867 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddee8e88-bf60-436d-a898-87aa5f35838e admin admin] PATH_INFO: `/v3/projects/93a42efeecef4a6e9da5a5d09dc5db9e/users/3ee52718ab8d4c7e910a1cb323334465/roles/39e5c666285a42fe8fc2c3afca6260d1` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:37:43.118307 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddee8e88-bf60-436d-a898-87aa5f35838e admin admin] RBAC: Authorizing `identity:create_grant(project_id=93a42efeecef4a6e9da5a5d09dc5db9e, user_id=3ee52718ab8d4c7e910a1cb323334465, role_id=39e5c666285a42fe8fc2c3afca6260d1)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:37:43.119200 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cbd567d-351e-436d-beec-8f1452136fe3 admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:37:43.121355 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddee8e88-bf60-436d-a898-87aa5f35838e admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:37:43.136851 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20586954-a817-4acd-9618-67ae664ec367 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:37:43.139740 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20586954-a817-4acd-9618-67ae664ec367 admin admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:37:43.140236 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20586954-a817-4acd-9618-67ae664ec367 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:37:43.140711 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20586954-a817-4acd-9618-67ae664ec367 admin admin] PATH_INFO: `/v3/roles` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:37:43.148911 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 951/1892] 10.222.0.33 () {60 vars in 1482 bytes} [Mon Jul 25 05:37:42 2022] PUT /identity/v3/projects/93a42efeecef4a6e9da5a5d09dc5db9e/users/3ee52718ab8d4c7e910a1cb323334465/roles/39e5c666285a42fe8fc2c3afca6260d1 => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 05:37:43.148911 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddee8e88-bf60-436d-a898-87aa5f35838e admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:37:43.153246 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-ddee8e88-bf60-436d-a898-87aa5f35838e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:37:43.156398 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20586954-a817-4acd-9618-67ae664ec367 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:37:43.158336 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20586954-a817-4acd-9618-67ae664ec367 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:37:43.176911 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 942/1893] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jul 25 05:37:42 2022] GET /identity/v3/roles => generated 1420 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 05:37:43.178081 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20586954-a817-4acd-9618-67ae664ec367 admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:37:43.194071 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20586954-a817-4acd-9618-67ae664ec367 admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:37:43.211313 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:37:43.286135 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddee8e88-bf60-436d-a898-87aa5f35838e admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:37:43.296958 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-433208ac-1957-4cd7-b149-f19aca1bcc2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:37:43.297855 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433208ac-1957-4cd7-b149-f19aca1bcc2c admin admin] REQUEST_METHOD: `PUT` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:37:43.299875 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433208ac-1957-4cd7-b149-f19aca1bcc2c admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:37:43.300172 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433208ac-1957-4cd7-b149-f19aca1bcc2c admin admin] PATH_INFO: `/v3/projects/d799bb4c8a1d45f8874976be48c31b3a/users/3e3a518b2cf54bf2b7dc8e84295782c2/roles/39e5c666285a42fe8fc2c3afca6260d1` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:37:43.306581 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-433208ac-1957-4cd7-b149-f19aca1bcc2c admin admin] RBAC: Authorizing `identity:create_grant(project_id=d799bb4c8a1d45f8874976be48c31b3a, user_id=3e3a518b2cf54bf2b7dc8e84295782c2, role_id=39e5c666285a42fe8fc2c3afca6260d1)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:37:43.315247 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-433208ac-1957-4cd7-b149-f19aca1bcc2c admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:37:43.325403 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5824943-5d7a-40b6-a45f-cf93f3ffadc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:37:43.326816 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5824943-5d7a-40b6-a45f-cf93f3ffadc3 admin admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:37:43.327143 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5824943-5d7a-40b6-a45f-cf93f3ffadc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:37:43.327637 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5824943-5d7a-40b6-a45f-cf93f3ffadc3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:37:43.328611 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5824943-5d7a-40b6-a45f-cf93f3ffadc3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:37:43.331736 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5824943-5d7a-40b6-a45f-cf93f3ffadc3 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:37:43.365517 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 952/1894] 10.222.0.33 () {60 vars in 1482 bytes} [Mon Jul 25 05:37:43 2022] PUT /identity/v3/projects/d799bb4c8a1d45f8874976be48c31b3a/users/3e3a518b2cf54bf2b7dc8e84295782c2/roles/39e5c666285a42fe8fc2c3afca6260d1 => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 05:37:43.366590 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-433208ac-1957-4cd7-b149-f19aca1bcc2c admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:37:43.371043 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 943/1895] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jul 25 05:37:43 2022] GET /identity/v3/roles => generated 1420 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 05:37:43.372834 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce60e98-10eb-4c64-83c6-15637b12a147 None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:37:43.373170 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce60e98-10eb-4c64-83c6-15637b12a147 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:37:43.373466 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce60e98-10eb-4c64-83c6-15637b12a147 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:37:43.424918 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-2ce60e98-10eb-4c64-83c6-15637b12a147 None None] MFA Rules not processed for user `3e3a518b2cf54bf2b7dc8e84295782c2`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:37:43.426302 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-2ce60e98-10eb-4c64-83c6-15637b12a147 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:37:43.469441 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-433208ac-1957-4cd7-b149-f19aca1bcc2c admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:37:43.484047 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57694dda-e87a-4c45-ac14-8c2c6eab437f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:37:43.485500 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57694dda-e87a-4c45-ac14-8c2c6eab437f admin admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:37:43.485789 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57694dda-e87a-4c45-ac14-8c2c6eab437f admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:37:43.486470 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57694dda-e87a-4c45-ac14-8c2c6eab437f admin admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:37:43.487525 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57694dda-e87a-4c45-ac14-8c2c6eab437f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:37:43.489248 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57694dda-e87a-4c45-ac14-8c2c6eab437f admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:37:43.504186 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 953/1896] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jul 25 05:37:43 2022] GET /identity/v3/roles => generated 1420 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 05:37:43.505568 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57694dda-e87a-4c45-ac14-8c2c6eab437f admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:37:43.519445 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57694dda-e87a-4c45-ac14-8c2c6eab437f admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:37:43.526708 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf5f942d-bddd-40f4-9de4-7cb68c4cd79b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:37:43.527803 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5f942d-bddd-40f4-9de4-7cb68c4cd79b admin admin] REQUEST_METHOD: `PUT` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:37:43.528606 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5f942d-bddd-40f4-9de4-7cb68c4cd79b admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:37:43.528926 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5f942d-bddd-40f4-9de4-7cb68c4cd79b admin admin] PATH_INFO: `/v3/projects/93a42efeecef4a6e9da5a5d09dc5db9e/users/3ee52718ab8d4c7e910a1cb323334465/roles/39e5c666285a42fe8fc2c3afca6260d1` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:37:43.531538 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf5f942d-bddd-40f4-9de4-7cb68c4cd79b admin admin] RBAC: Authorizing `identity:create_grant(project_id=93a42efeecef4a6e9da5a5d09dc5db9e, user_id=3ee52718ab8d4c7e910a1cb323334465, role_id=39e5c666285a42fe8fc2c3afca6260d1)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:37:43.534742 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf5f942d-bddd-40f4-9de4-7cb68c4cd79b admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:37:43.548843 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 954/1897] 10.222.0.33 () {60 vars in 1482 bytes} [Mon Jul 25 05:37:43 2022] PUT /identity/v3/projects/93a42efeecef4a6e9da5a5d09dc5db9e/users/3ee52718ab8d4c7e910a1cb323334465/roles/39e5c666285a42fe8fc2c3afca6260d1 => generated 0 bytes in 44 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 05:37:43.550080 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3ea08e-18d4-4c68-8cdf-525a357fcd67 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:37:43.550370 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3ea08e-18d4-4c68-8cdf-525a357fcd67 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:37:43.550811 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3ea08e-18d4-4c68-8cdf-525a357fcd67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:37:43.566911 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:DELETE=2,keystone:SELECT=97,keystone:INSERT=1 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:37:43.584580 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 944/1898] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 25 05:37:43 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:37:43.584580 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce60e98-10eb-4c64-83c6-15637b12a147 None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:37:43.605151 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce60e98-10eb-4c64-83c6-15637b12a147 None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:37:43.606393 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-6c3ea08e-18d4-4c68-8cdf-525a357fcd67 None None] MFA Rules not processed for user `be5eb59db01b48448fef59467ee778ca`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:37:43.608412 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-6c3ea08e-18d4-4c68-8cdf-525a357fcd67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:37:43.725263 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e205a2-e318-45c5-bb79-ce7a949b1191 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:37:43.726211 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e205a2-e318-45c5-bb79-ce7a949b1191 admin admin] REQUEST_METHOD: `PUT` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:37:43.726621 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e205a2-e318-45c5-bb79-ce7a949b1191 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:37:43.726960 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e205a2-e318-45c5-bb79-ce7a949b1191 admin admin] PATH_INFO: `/v3/projects/87bf5735962e48ccbd32293dc0d4857e/users/31a48da0425f41008fd4faba49d9cb4b/roles/5074e74e35f54d4b9bb415c34999a82a` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:37:43.747248 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41e205a2-e318-45c5-bb79-ce7a949b1191 admin admin] RBAC: Authorizing `identity:create_grant(project_id=87bf5735962e48ccbd32293dc0d4857e, user_id=31a48da0425f41008fd4faba49d9cb4b, role_id=5074e74e35f54d4b9bb415c34999a82a)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:37:43.750818 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 955/1899] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 25 05:37:43 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:37:43.754342 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41e205a2-e318-45c5-bb79-ce7a949b1191 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:37:43.756741 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dcb2bd6-dc35-44da-bbdc-ed09c249ed63 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:37:43.757521 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dcb2bd6-dc35-44da-bbdc-ed09c249ed63 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:37:43.757521 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dcb2bd6-dc35-44da-bbdc-ed09c249ed63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:37:43.780219 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 945/1900] 10.222.0.33 () {60 vars in 1482 bytes} [Mon Jul 25 05:37:43 2022] PUT /identity/v3/projects/87bf5735962e48ccbd32293dc0d4857e/users/31a48da0425f41008fd4faba49d9cb4b/roles/5074e74e35f54d4b9bb415c34999a82a => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 05:37:43.792424 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e205a2-e318-45c5-bb79-ce7a949b1191 admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:37:43.794194 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-41e205a2-e318-45c5-bb79-ce7a949b1191 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:37:43.828393 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-3dcb2bd6-dc35-44da-bbdc-ed09c249ed63 None None] MFA Rules not processed for user `3ee52718ab8d4c7e910a1cb323334465`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:37:43.830446 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-3dcb2bd6-dc35-44da-bbdc-ed09c249ed63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:37:43.871000 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:37:43.922269 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e205a2-e318-45c5-bb79-ce7a949b1191 admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:37:43.933382 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdcfb8d2-500c-4f66-84e1-ce95b1877f77 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db57b478c25643dbaf936f90f6e17d0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92afba266cce4d1bad4134c351e74be3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:37:43.935767 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdcfb8d2-500c-4f66-84e1-ce95b1877f77 service cinder] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:37:43.939907 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdcfb8d2-500c-4f66-84e1-ce95b1877f77 service cinder] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:37:43.940526 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdcfb8d2-500c-4f66-84e1-ce95b1877f77 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:37:43.995160 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 956/1901] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 25 05:37:43 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 239 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:37:43.996474 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dcb2bd6-dc35-44da-bbdc-ed09c249ed63 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:37:43.999174 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-3dcb2bd6-dc35-44da-bbdc-ed09c249ed63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:37:44.068455 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdcfb8d2-500c-4f66-84e1-ce95b1877f77 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:37:44.071686 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdcfb8d2-500c-4f66-84e1-ce95b1877f77 service cinder] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:37:44.094282 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 946/1902] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jul 25 05:37:43 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 05:37:44.095465 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fdd6888-f4c6-4fb0-8b59-de6713a2beec None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:37:44.095774 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fdd6888-f4c6-4fb0-8b59-de6713a2beec None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:37:44.096064 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fdd6888-f4c6-4fb0-8b59-de6713a2beec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:37:44.132153 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dcb2bd6-dc35-44da-bbdc-ed09c249ed63 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:37:44.149799 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4800f09f-0cd3-444f-b8a0-d66df7ba9e74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:37:44.151302 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4800f09f-0cd3-444f-b8a0-d66df7ba9e74 admin admin] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:37:44.154481 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4800f09f-0cd3-444f-b8a0-d66df7ba9e74 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:37:44.154895 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4800f09f-0cd3-444f-b8a0-d66df7ba9e74 admin admin] PATH_INFO: `/v3/projects` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:37:44.155867 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4800f09f-0cd3-444f-b8a0-d66df7ba9e74 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:37:44.158313 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4800f09f-0cd3-444f-b8a0-d66df7ba9e74 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:37:44.191260 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-8fdd6888-f4c6-4fb0-8b59-de6713a2beec None None] MFA Rules not processed for user `31a48da0425f41008fd4faba49d9cb4b`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:37:44.192600 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-8fdd6888-f4c6-4fb0-8b59-de6713a2beec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:37:44.221775 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:37:44.248335 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 957/1903] 10.222.0.33 () {60 vars in 1152 bytes} [Mon Jul 25 05:37:43 2022] POST /identity/v3/projects => generated 371 bytes in 252 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 05:37:44.249517 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4800f09f-0cd3-444f-b8a0-d66df7ba9e74 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:37:44.254220 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-4800f09f-0cd3-444f-b8a0-d66df7ba9e74 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:37:44.314173 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 947/1904] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 25 05:37:44 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 220 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:37:44.315181 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fdd6888-f4c6-4fb0-8b59-de6713a2beec None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:37:44.316471 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-8fdd6888-f4c6-4fb0-8b59-de6713a2beec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:37:44.329869 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fdd6888-f4c6-4fb0-8b59-de6713a2beec None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:37:44.388007 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4800f09f-0cd3-444f-b8a0-d66df7ba9e74 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:37:44.400807 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f034198f-fd58-40e0-ae53-2cb692521ca9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db57b478c25643dbaf936f90f6e17d0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92afba266cce4d1bad4134c351e74be3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:37:44.402398 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f034198f-fd58-40e0-ae53-2cb692521ca9 service cinder] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:37:44.403242 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f034198f-fd58-40e0-ae53-2cb692521ca9 service cinder] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:37:44.403531 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f034198f-fd58-40e0-ae53-2cb692521ca9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:37:44.405985 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4617713-ca28-40f6-b2bf-3c7f53dae278 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:37:44.406873 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4617713-ca28-40f6-b2bf-3c7f53dae278 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:37:44.407155 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4617713-ca28-40f6-b2bf-3c7f53dae278 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:37:44.407436 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4617713-ca28-40f6-b2bf-3c7f53dae278 admin admin] PATH_INFO: `/v3/projects/a15b25ea642841e3a294e3f12c500c76` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:37:44.410554 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4617713-ca28-40f6-b2bf-3c7f53dae278 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a15b25ea642841e3a294e3f12c500c76)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:37:44.414676 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4617713-ca28-40f6-b2bf-3c7f53dae278 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:37:44.527615 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-f4617713-ca28-40f6-b2bf-3c7f53dae278 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a15b25ea642841e3a294e3f12c500c76', 'request_id': 'req-f4617713-ca28-40f6-b2bf-3c7f53dae278'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:37:44.529171 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=95,keystone:INSERT=1,keystone:DELETE=4 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:37:44.540325 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-f4617713-ca28-40f6-b2bf-3c7f53dae278 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a15b25ea642841e3a294e3f12c500c76', 'request_id': 'req-f4617713-ca28-40f6-b2bf-3c7f53dae278'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:37:44.553602 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-f4617713-ca28-40f6-b2bf-3c7f53dae278 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a15b25ea642841e3a294e3f12c500c76', 'request_id': 'req-f4617713-ca28-40f6-b2bf-3c7f53dae278'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:37:44.558053 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f034198f-fd58-40e0-ae53-2cb692521ca9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:37:44.560063 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f034198f-fd58-40e0-ae53-2cb692521ca9 service cinder] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:37:44.576892 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 948/1905] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jul 25 05:37:44 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 262 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 05:37:44.577585 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-f4617713-ca28-40f6-b2bf-3c7f53dae278 admin admin] The token cache is being invalidate because project a15b25ea642841e3a294e3f12c500c76 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80249) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 05:37:44.577884 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-f4617713-ca28-40f6-b2bf-3c7f53dae278 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:37:44.578596 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-f4617713-ca28-40f6-b2bf-3c7f53dae278 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:37:44.580685 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 958/1906] 10.222.0.33 () {58 vars in 1232 bytes} [Mon Jul 25 05:37:44 2022] DELETE /identity/v3/projects/a15b25ea642841e3a294e3f12c500c76 => generated 0 bytes in 331 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 05:37:44.581587 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4617713-ca28-40f6-b2bf-3c7f53dae278 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:37:44.585355 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-f4617713-ca28-40f6-b2bf-3c7f53dae278 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:37:44.591755 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f034198f-fd58-40e0-ae53-2cb692521ca9 service cinder] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:37:44.608946 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f034198f-fd58-40e0-ae53-2cb692521ca9 service cinder] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:37:44.711809 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4617713-ca28-40f6-b2bf-3c7f53dae278 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:37:44.725900 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3138c928-db3a-4f4e-8669-377402a71b76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:37:44.726780 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3138c928-db3a-4f4e-8669-377402a71b76 admin admin] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:37:44.727072 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3138c928-db3a-4f4e-8669-377402a71b76 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:37:44.728806 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3138c928-db3a-4f4e-8669-377402a71b76 admin admin] PATH_INFO: `/v3/users` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:37:44.729292 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3138c928-db3a-4f4e-8669-377402a71b76 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:37:44.734637 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3138c928-db3a-4f4e-8669-377402a71b76 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:37:44.773418 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-766849a6-a831-46ff-ac9f-06fceda3a3f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:37:44.774367 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766849a6-a831-46ff-ac9f-06fceda3a3f4 admin admin] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:37:44.774850 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766849a6-a831-46ff-ac9f-06fceda3a3f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:37:44.775549 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766849a6-a831-46ff-ac9f-06fceda3a3f4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:37:44.777997 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-766849a6-a831-46ff-ac9f-06fceda3a3f4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:37:44.780812 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-766849a6-a831-46ff-ac9f-06fceda3a3f4 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:37:44.795880 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 959/1907] 10.222.0.33 () {60 vars in 1143 bytes} [Mon Jul 25 05:37:44 2022] POST /identity/v3/users => generated 292 bytes in 214 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 05:37:44.808682 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:37:44.810017 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f397640b-581a-4781-9ca3-4e3e49af6986 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:37:44.810317 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f397640b-581a-4781-9ca3-4e3e49af6986 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:37:44.810764 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f397640b-581a-4781-9ca3-4e3e49af6986 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:37:44.841886 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 949/1908] 10.222.0.33 () {60 vars in 1152 bytes} [Mon Jul 25 05:37:44 2022] POST /identity/v3/projects => generated 375 bytes in 247 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 05:37:44.842167 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-766849a6-a831-46ff-ac9f-06fceda3a3f4 admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:37:44.845431 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-766849a6-a831-46ff-ac9f-06fceda3a3f4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:37:44.849175 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-f397640b-581a-4781-9ca3-4e3e49af6986 None None] MFA Rules not processed for user `1eae540242d94c10b0ad33960a855f08`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:37:44.850487 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-f397640b-581a-4781-9ca3-4e3e49af6986 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:37:44.920962 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 960/1909] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 25 05:37:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:37:44.921935 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f397640b-581a-4781-9ca3-4e3e49af6986 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:37:44.939455 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f397640b-581a-4781-9ca3-4e3e49af6986 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:37:44.940998 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-766849a6-a831-46ff-ac9f-06fceda3a3f4 admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:37:44.947130 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c2da823-b324-4ef4-9d4a-7b2ed9cb7567 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:37:44.948246 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2da823-b324-4ef4-9d4a-7b2ed9cb7567 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:37:44.948540 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2da823-b324-4ef4-9d4a-7b2ed9cb7567 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:37:44.948854 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2da823-b324-4ef4-9d4a-7b2ed9cb7567 admin admin] PATH_INFO: `/v3/users/0d6817b3321748b59a8006cc96b63633` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:37:44.950143 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c2da823-b324-4ef4-9d4a-7b2ed9cb7567 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0d6817b3321748b59a8006cc96b63633)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:37:44.952305 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c2da823-b324-4ef4-9d4a-7b2ed9cb7567 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:37:44.955049 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d25e1ebb-bc75-4727-85ac-cfbedd74e58a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:37:44.955873 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25e1ebb-bc75-4727-85ac-cfbedd74e58a admin admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:37:44.956269 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25e1ebb-bc75-4727-85ac-cfbedd74e58a admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:37:44.956674 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25e1ebb-bc75-4727-85ac-cfbedd74e58a admin admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:37:44.957658 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d25e1ebb-bc75-4727-85ac-cfbedd74e58a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:37:44.960591 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d25e1ebb-bc75-4727-85ac-cfbedd74e58a admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:37:44.975258 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 961/1910] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jul 25 05:37:44 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 05:37:44.976421 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d25e1ebb-bc75-4727-85ac-cfbedd74e58a admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:37:44.978694 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-d25e1ebb-bc75-4727-85ac-cfbedd74e58a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:37:45.000068 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d25e1ebb-bc75-4727-85ac-cfbedd74e58a admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:37:45.007294 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b4c440c-d8b1-4f9c-81c4-9375327e7b4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:37:45.008332 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4c440c-d8b1-4f9c-81c4-9375327e7b4b admin admin] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:37:45.008843 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4c440c-d8b1-4f9c-81c4-9375327e7b4b admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:37:45.009566 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4c440c-d8b1-4f9c-81c4-9375327e7b4b admin admin] PATH_INFO: `/v3/users` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:37:45.011300 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b4c440c-d8b1-4f9c-81c4-9375327e7b4b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:37:45.014560 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b4c440c-d8b1-4f9c-81c4-9375327e7b4b admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:37:45.030028 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-4c2da823-b324-4ef4-9d4a-7b2ed9cb7567 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0d6817b3321748b59a8006cc96b63633'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:37:45.048961 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 962/1911] 10.222.0.33 () {60 vars in 1143 bytes} [Mon Jul 25 05:37:44 2022] POST /identity/v3/users => generated 294 bytes in 73 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 05:37:45.050224 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b4c440c-d8b1-4f9c-81c4-9375327e7b4b admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:37:45.054014 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-1b4c440c-d8b1-4f9c-81c4-9375327e7b4b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:37:45.054167 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-4c2da823-b324-4ef4-9d4a-7b2ed9cb7567 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0d6817b3321748b59a8006cc96b63633'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:37:45.054848 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-4c2da823-b324-4ef4-9d4a-7b2ed9cb7567 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0d6817b3321748b59a8006cc96b63633'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:37:45.068808 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-4c2da823-b324-4ef4-9d4a-7b2ed9cb7567 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0d6817b3321748b59a8006cc96b63633'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:37:45.070793 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-4c2da823-b324-4ef4-9d4a-7b2ed9cb7567 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0d6817b3321748b59a8006cc96b63633'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:37:45.071166 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b4c440c-d8b1-4f9c-81c4-9375327e7b4b admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:37:45.078586 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8828c644-6f30-4b00-b753-9b7372ec85aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:37:45.081029 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8828c644-6f30-4b00-b753-9b7372ec85aa admin admin] REQUEST_METHOD: `DELETE` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:37:45.081327 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8828c644-6f30-4b00-b753-9b7372ec85aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:37:45.081614 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8828c644-6f30-4b00-b753-9b7372ec85aa admin admin] PATH_INFO: `/v3/users/a42f1441b8564b5a82c79798deb910d5` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:37:45.083399 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8828c644-6f30-4b00-b753-9b7372ec85aa admin admin] RBAC: Authorizing `identity:delete_user(user_id=a42f1441b8564b5a82c79798deb910d5)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:37:45.086542 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8828c644-6f30-4b00-b753-9b7372ec85aa admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:37:45.091478 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 950/1912] 10.222.0.33 () {58 vars in 1223 bytes} [Mon Jul 25 05:37:44 2022] DELETE /identity/v3/users/0d6817b3321748b59a8006cc96b63633 => generated 0 bytes in 251 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 05:37:45.093603 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba10a19-b11f-42e1-a06b-7e24892de497 None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:37:45.094315 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba10a19-b11f-42e1-a06b-7e24892de497 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:37:45.094617 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba10a19-b11f-42e1-a06b-7e24892de497 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:37:45.120128 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-6ba10a19-b11f-42e1-a06b-7e24892de497 None None] MFA Rules not processed for user `1eae540242d94c10b0ad33960a855f08`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:37:45.122103 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-6ba10a19-b11f-42e1-a06b-7e24892de497 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:37:45.187065 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:DELETE=8,keystone:SELECT=85,keystone:INSERT=7 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:37:45.208545 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-8828c644-6f30-4b00-b753-9b7372ec85aa admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a42f1441b8564b5a82c79798deb910d5'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:37:45.209690 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-8828c644-6f30-4b00-b753-9b7372ec85aa admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a42f1441b8564b5a82c79798deb910d5'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:37:45.226973 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-8828c644-6f30-4b00-b753-9b7372ec85aa admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a42f1441b8564b5a82c79798deb910d5'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:37:45.253284 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-8828c644-6f30-4b00-b753-9b7372ec85aa admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a42f1441b8564b5a82c79798deb910d5'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:37:45.254275 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-8828c644-6f30-4b00-b753-9b7372ec85aa admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a42f1441b8564b5a82c79798deb910d5'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:37:45.262624 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 951/1913] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 25 05:37:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:37:45.265878 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbcc0212-c349-40a8-b681-72d50ee8ed45 None None] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:37:45.265878 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbcc0212-c349-40a8-b681-72d50ee8ed45 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:37:45.265878 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbcc0212-c349-40a8-b681-72d50ee8ed45 None None] PATH_INFO: `/` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:37:45.267973 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 952/1914] 10.222.0.33 () {54 vars in 921 bytes} [Mon Jul 25 05:37:45 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:37:45.268968 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbcc0212-c349-40a8-b681-72d50ee8ed45 None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:37:45.271498 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-cbcc0212-c349-40a8-b681-72d50ee8ed45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:37:45.286694 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 963/1915] 10.222.0.33 () {58 vars in 1223 bytes} [Mon Jul 25 05:37:45 2022] DELETE /identity/v3/users/a42f1441b8564b5a82c79798deb910d5 => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 05:37:45.287860 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8828c644-6f30-4b00-b753-9b7372ec85aa admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:37:45.292426 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-8828c644-6f30-4b00-b753-9b7372ec85aa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:37:45.292820 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbcc0212-c349-40a8-b681-72d50ee8ed45 None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:37:45.309760 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:INSERT=2,keystone:SELECT=88,keystone:DELETE=10 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:37:45.387881 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c66c4b-297e-44cb-97b6-2901eb1ad22a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:37:45.389705 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c66c4b-297e-44cb-97b6-2901eb1ad22a admin admin] REQUEST_METHOD: `PUT` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:37:45.390346 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c66c4b-297e-44cb-97b6-2901eb1ad22a admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:37:45.392927 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c66c4b-297e-44cb-97b6-2901eb1ad22a admin admin] PATH_INFO: `/v3/projects/d9121e3b20714e6bb1930dd2d0d6be96/users/357148a4c5564dc3ab9b2273513ec470/roles/39e5c666285a42fe8fc2c3afca6260d1` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:37:45.429818 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7c66c4b-297e-44cb-97b6-2901eb1ad22a admin admin] RBAC: Authorizing `identity:create_grant(project_id=d9121e3b20714e6bb1930dd2d0d6be96, user_id=357148a4c5564dc3ab9b2273513ec470, role_id=39e5c666285a42fe8fc2c3afca6260d1)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:37:45.436691 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8828c644-6f30-4b00-b753-9b7372ec85aa admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:37:45.439740 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7c66c4b-297e-44cb-97b6-2901eb1ad22a admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:37:45.459330 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-275557e3-86f6-44c2-a249-2a8018bce7c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:37:45.460062 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275557e3-86f6-44c2-a249-2a8018bce7c8 admin admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:37:45.460999 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275557e3-86f6-44c2-a249-2a8018bce7c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:37:45.461283 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275557e3-86f6-44c2-a249-2a8018bce7c8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:37:45.465738 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-275557e3-86f6-44c2-a249-2a8018bce7c8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:37:45.467414 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-275557e3-86f6-44c2-a249-2a8018bce7c8 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:37:45.474601 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 953/1916] 10.222.0.33 () {60 vars in 1482 bytes} [Mon Jul 25 05:37:45 2022] PUT /identity/v3/projects/d9121e3b20714e6bb1930dd2d0d6be96/users/357148a4c5564dc3ab9b2273513ec470/roles/39e5c666285a42fe8fc2c3afca6260d1 => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 05:37:45.478398 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e777b00-7bad-4625-8b80-fb4426796229 None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:37:45.478955 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e777b00-7bad-4625-8b80-fb4426796229 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:37:45.480003 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e777b00-7bad-4625-8b80-fb4426796229 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:37:45.495930 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 964/1917] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jul 25 05:37:45 2022] GET /identity/v3/roles => generated 1420 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 05:37:45.496892 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-275557e3-86f6-44c2-a249-2a8018bce7c8 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:37:45.506799 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-6e777b00-7bad-4625-8b80-fb4426796229 None None] MFA Rules not processed for user `1eae540242d94c10b0ad33960a855f08`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:37:45.507692 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-275557e3-86f6-44c2-a249-2a8018bce7c8 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:37:45.508529 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-6e777b00-7bad-4625-8b80-fb4426796229 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:37:45.596633 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 954/1918] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 25 05:37:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:37:45.597568 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74dbc6d-f32b-42fa-b156-675b4c35d179 None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:37:45.597866 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74dbc6d-f32b-42fa-b156-675b4c35d179 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:37:45.598147 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74dbc6d-f32b-42fa-b156-675b4c35d179 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:37:45.617022 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb90a2a-77ab-4f85-941a-33dd7da1144e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:37:45.617709 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb90a2a-77ab-4f85-941a-33dd7da1144e admin admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:37:45.618042 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb90a2a-77ab-4f85-941a-33dd7da1144e admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:37:45.618278 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb90a2a-77ab-4f85-941a-33dd7da1144e admin admin] PATH_INFO: `/v3/domains` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:37:45.619123 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccb90a2a-77ab-4f85-941a-33dd7da1144e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:37:45.619481 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccb90a2a-77ab-4f85-941a-33dd7da1144e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:37:45.621415 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-d74dbc6d-f32b-42fa-b156-675b4c35d179 None None] MFA Rules not processed for user `1eae540242d94c10b0ad33960a855f08`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:37:45.621718 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccb90a2a-77ab-4f85-941a-33dd7da1144e admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:37:45.622659 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-d74dbc6d-f32b-42fa-b156-675b4c35d179 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:37:45.632948 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 955/1919] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 25 05:37:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:37:45.634007 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9220cbe4-b6f6-44c2-ab04-67f210e206c2 None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:37:45.634359 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9220cbe4-b6f6-44c2-ab04-67f210e206c2 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:37:45.634579 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9220cbe4-b6f6-44c2-ab04-67f210e206c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:37:45.640290 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 965/1920] 10.222.0.33 () {58 vars in 1152 bytes} [Mon Jul 25 05:37:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:37:45.642086 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb90a2a-77ab-4f85-941a-33dd7da1144e admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:37:45.654632 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb90a2a-77ab-4f85-941a-33dd7da1144e admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:37:45.658167 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-9220cbe4-b6f6-44c2-ab04-67f210e206c2 None None] MFA Rules not processed for user `1cd482a61e8a420db45bb532e5dae6e2`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:37:45.659417 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-9220cbe4-b6f6-44c2-ab04-67f210e206c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:37:45.659640 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f1d226-118e-4fd6-984a-c4bc8bd427d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:37:45.660352 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f1d226-118e-4fd6-984a-c4bc8bd427d6 admin admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:37:45.661347 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f1d226-118e-4fd6-984a-c4bc8bd427d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:37:45.661347 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f1d226-118e-4fd6-984a-c4bc8bd427d6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:37:45.662421 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8f1d226-118e-4fd6-984a-c4bc8bd427d6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:37:45.664032 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8f1d226-118e-4fd6-984a-c4bc8bd427d6 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:37:45.676639 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 966/1921] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jul 25 05:37:45 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 05:37:45.677514 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f1d226-118e-4fd6-984a-c4bc8bd427d6 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:37:45.689449 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f1d226-118e-4fd6-984a-c4bc8bd427d6 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:37:45.696017 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c98ff5-ef7b-42e7-8019-31a9db93c7e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:37:45.697255 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c98ff5-ef7b-42e7-8019-31a9db93c7e4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:37:45.697255 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c98ff5-ef7b-42e7-8019-31a9db93c7e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:37:45.697489 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c98ff5-ef7b-42e7-8019-31a9db93c7e4 admin admin] PATH_INFO: `/v3/projects/ad8eaf4b84a747e19aba9dd2737e9541/users/6d398ae141d7415082da8e61051677b1/roles/39e5c666285a42fe8fc2c3afca6260d1` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:37:45.715953 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7c98ff5-ef7b-42e7-8019-31a9db93c7e4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ad8eaf4b84a747e19aba9dd2737e9541, user_id=6d398ae141d7415082da8e61051677b1, role_id=39e5c666285a42fe8fc2c3afca6260d1)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:37:45.719756 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7c98ff5-ef7b-42e7-8019-31a9db93c7e4 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:37:45.736591 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 967/1922] 10.222.0.33 () {60 vars in 1482 bytes} [Mon Jul 25 05:37:45 2022] PUT /identity/v3/projects/ad8eaf4b84a747e19aba9dd2737e9541/users/6d398ae141d7415082da8e61051677b1/roles/39e5c666285a42fe8fc2c3afca6260d1 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 05:37:45.737488 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c98ff5-ef7b-42e7-8019-31a9db93c7e4 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:37:45.739054 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-a7c98ff5-ef7b-42e7-8019-31a9db93c7e4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:37:45.741174 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 956/1923] 10.222.0.33 () {58 vars in 1018 bytes} [Mon Jul 25 05:37:45 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:37:45.741977 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9220cbe4-b6f6-44c2-ab04-67f210e206c2 None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:37:45.751238 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=93,keystone:DELETE=5,keystone:INSERT=2 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:37:45.758434 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9220cbe4-b6f6-44c2-ab04-67f210e206c2 None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:37:45.783219 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:37:45.803923 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c98ff5-ef7b-42e7-8019-31a9db93c7e4 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:37:45.809341 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63492e98-f919-4546-b755-df5eee7cb1e6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b2931200f99e43d79a50a9f6e02738e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92afba266cce4d1bad4134c351e74be3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:37:45.810102 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63492e98-f919-4546-b755-df5eee7cb1e6 service neutron] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:37:45.810463 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63492e98-f919-4546-b755-df5eee7cb1e6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:37:45.810815 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63492e98-f919-4546-b755-df5eee7cb1e6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:37:45.833715 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ed8c2f-d641-49dc-8c70-549c60f99a1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:37:45.834454 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ed8c2f-d641-49dc-8c70-549c60f99a1f admin admin] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:37:45.834746 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ed8c2f-d641-49dc-8c70-549c60f99a1f admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:37:45.835028 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ed8c2f-d641-49dc-8c70-549c60f99a1f admin admin] PATH_INFO: `/v3/projects` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:37:45.835732 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20ed8c2f-d641-49dc-8c70-549c60f99a1f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:37:45.837676 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20ed8c2f-d641-49dc-8c70-549c60f99a1f admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:37:45.870651 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 957/1924] 10.222.0.33 () {60 vars in 1152 bytes} [Mon Jul 25 05:37:45 2022] POST /identity/v3/projects => generated 383 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 05:37:45.870651 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ed8c2f-d641-49dc-8c70-549c60f99a1f admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:37:45.892060 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63492e98-f919-4546-b755-df5eee7cb1e6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:37:45.893983 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63492e98-f919-4546-b755-df5eee7cb1e6 service neutron] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:37:45.910618 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 968/1925] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jul 25 05:37:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 173 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 05:37:45.911658 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63492e98-f919-4546-b755-df5eee7cb1e6 service neutron] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:37:45.925882 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63492e98-f919-4546-b755-df5eee7cb1e6 service neutron] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:37:45.947131 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ed8c2f-d641-49dc-8c70-549c60f99a1f admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:37:45.952491 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea4ea6ab-c9be-428c-a123-ac1db7ed7530 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b2931200f99e43d79a50a9f6e02738e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92afba266cce4d1bad4134c351e74be3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:37:45.953397 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4ea6ab-c9be-428c-a123-ac1db7ed7530 service neutron] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:37:45.954929 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4ea6ab-c9be-428c-a123-ac1db7ed7530 service neutron] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:37:45.955470 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4ea6ab-c9be-428c-a123-ac1db7ed7530 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:37:46.028990 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58515009-d2b8-470c-972c-37361634797d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:37:46.029905 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58515009-d2b8-470c-972c-37361634797d admin admin] REQUEST_METHOD: `PUT` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:37:46.030198 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58515009-d2b8-470c-972c-37361634797d admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:37:46.030479 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58515009-d2b8-470c-972c-37361634797d admin admin] PATH_INFO: `/v3/projects/d9121e3b20714e6bb1930dd2d0d6be96/users/357148a4c5564dc3ab9b2273513ec470/roles/39e5c666285a42fe8fc2c3afca6260d1` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:37:46.032723 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58515009-d2b8-470c-972c-37361634797d admin admin] RBAC: Authorizing `identity:create_grant(project_id=d9121e3b20714e6bb1930dd2d0d6be96, user_id=357148a4c5564dc3ab9b2273513ec470, role_id=39e5c666285a42fe8fc2c3afca6260d1)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:37:46.036095 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58515009-d2b8-470c-972c-37361634797d admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:37:46.050398 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 969/1926] 10.222.0.33 () {60 vars in 1482 bytes} [Mon Jul 25 05:37:45 2022] PUT /identity/v3/projects/d9121e3b20714e6bb1930dd2d0d6be96/users/357148a4c5564dc3ab9b2273513ec470/roles/39e5c666285a42fe8fc2c3afca6260d1 => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 05:37:46.052416 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea4ea6ab-c9be-428c-a123-ac1db7ed7530 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:37:46.054140 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea4ea6ab-c9be-428c-a123-ac1db7ed7530 service neutron] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:37:46.055353 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58515009-d2b8-470c-972c-37361634797d admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:37:46.067978 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 958/1927] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jul 25 05:37:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 200 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 05:37:46.069023 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea4ea6ab-c9be-428c-a123-ac1db7ed7530 service neutron] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:37:46.071068 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-ea4ea6ab-c9be-428c-a123-ac1db7ed7530 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:37:46.140889 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:37:46.166049 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:37:46.195959 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58515009-d2b8-470c-972c-37361634797d admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:37:46.216169 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea4ea6ab-c9be-428c-a123-ac1db7ed7530 service neutron] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:37:46.218028 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab50128b-6b19-457e-ad11-e68108f5d2a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:37:46.220620 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab50128b-6b19-457e-ad11-e68108f5d2a8 admin admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:37:46.222412 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab50128b-6b19-457e-ad11-e68108f5d2a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:37:46.223119 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab50128b-6b19-457e-ad11-e68108f5d2a8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:37:46.226031 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab50128b-6b19-457e-ad11-e68108f5d2a8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:37:46.232125 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab50128b-6b19-457e-ad11-e68108f5d2a8 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:37:46.267430 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 970/1928] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jul 25 05:37:46 2022] GET /identity/v3/roles => generated 1420 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 05:37:46.269199 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab50128b-6b19-457e-ad11-e68108f5d2a8 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:37:46.314350 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab50128b-6b19-457e-ad11-e68108f5d2a8 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:37:46.334059 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ad44e3-c470-492c-8b3b-a51826cd6da6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cd482a61e8a420db45bb532e5dae6e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92afba266cce4d1bad4134c351e74be3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:37:46.334953 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ad44e3-c470-492c-8b3b-a51826cd6da6 service nova] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:37:46.335245 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ad44e3-c470-492c-8b3b-a51826cd6da6 service nova] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:37:46.335571 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ad44e3-c470-492c-8b3b-a51826cd6da6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:37:46.335742 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e01faad2-b4a4-49b8-8eb4-a5b1e4da0a86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:37:46.337585 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01faad2-b4a4-49b8-8eb4-a5b1e4da0a86 admin admin] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:37:46.338922 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01faad2-b4a4-49b8-8eb4-a5b1e4da0a86 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:37:46.340375 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01faad2-b4a4-49b8-8eb4-a5b1e4da0a86 admin admin] PATH_INFO: `/v3/users` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:37:46.342738 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e01faad2-b4a4-49b8-8eb4-a5b1e4da0a86 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:37:46.346317 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5ad44e3-c470-492c-8b3b-a51826cd6da6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:37:46.347110 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e01faad2-b4a4-49b8-8eb4-a5b1e4da0a86 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:37:46.348431 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5ad44e3-c470-492c-8b3b-a51826cd6da6 service nova] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:37:46.362421 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 959/1929] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jul 25 05:37:46 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 294 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 05:37:46.363966 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ad44e3-c470-492c-8b3b-a51826cd6da6 service nova] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:37:46.381080 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ad44e3-c470-492c-8b3b-a51826cd6da6 service nova] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:37:46.387583 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00157ba3-f5fe-4bf6-a894-6b76b8ec79e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:37:46.388583 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00157ba3-f5fe-4bf6-a894-6b76b8ec79e7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:37:46.388904 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00157ba3-f5fe-4bf6-a894-6b76b8ec79e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:37:46.389435 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 971/1930] 10.222.0.33 () {60 vars in 1143 bytes} [Mon Jul 25 05:37:46 2022] POST /identity/v3/users => generated 292 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 05:37:46.392291 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00157ba3-f5fe-4bf6-a894-6b76b8ec79e7 admin admin] PATH_INFO: `/v3/projects/ad8eaf4b84a747e19aba9dd2737e9541/users/6d398ae141d7415082da8e61051677b1/roles/39e5c666285a42fe8fc2c3afca6260d1` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:37:46.393715 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e01faad2-b4a4-49b8-8eb4-a5b1e4da0a86 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:37:46.394759 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00157ba3-f5fe-4bf6-a894-6b76b8ec79e7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ad8eaf4b84a747e19aba9dd2737e9541, user_id=6d398ae141d7415082da8e61051677b1, role_id=39e5c666285a42fe8fc2c3afca6260d1)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:37:46.398688 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00157ba3-f5fe-4bf6-a894-6b76b8ec79e7 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:37:46.407497 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e01faad2-b4a4-49b8-8eb4-a5b1e4da0a86 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:37:46.413919 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 960/1931] 10.222.0.33 () {60 vars in 1482 bytes} [Mon Jul 25 05:37:46 2022] PUT /identity/v3/projects/ad8eaf4b84a747e19aba9dd2737e9541/users/6d398ae141d7415082da8e61051677b1/roles/39e5c666285a42fe8fc2c3afca6260d1 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 05:37:46.426371 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174363ea-f025-44b0-bdc6-9e3fd55116fe None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:37:46.426671 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174363ea-f025-44b0-bdc6-9e3fd55116fe None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:37:46.426958 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174363ea-f025-44b0-bdc6-9e3fd55116fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:37:46.496369 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-174363ea-f025-44b0-bdc6-9e3fd55116fe None None] MFA Rules not processed for user `6d398ae141d7415082da8e61051677b1`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:37:46.497823 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-174363ea-f025-44b0-bdc6-9e3fd55116fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:37:46.515122 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ea4bb4-de35-4f2f-892b-1bb2db54fe4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:37:46.515993 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ea4bb4-de35-4f2f-892b-1bb2db54fe4b admin admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:37:46.516660 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ea4bb4-de35-4f2f-892b-1bb2db54fe4b admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:37:46.517122 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ea4bb4-de35-4f2f-892b-1bb2db54fe4b admin admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:37:46.519010 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58ea4bb4-de35-4f2f-892b-1bb2db54fe4b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:37:46.521189 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58ea4bb4-de35-4f2f-892b-1bb2db54fe4b admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:37:46.541340 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 972/1932] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jul 25 05:37:46 2022] GET /identity/v3/roles => generated 1420 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 05:37:46.542731 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ea4bb4-de35-4f2f-892b-1bb2db54fe4b admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:37:46.544675 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-58ea4bb4-de35-4f2f-892b-1bb2db54fe4b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:37:46.559796 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ea4bb4-de35-4f2f-892b-1bb2db54fe4b admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:37:46.565608 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cede0e13-0b73-420d-95c5-01883b11c6f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:37:46.566717 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cede0e13-0b73-420d-95c5-01883b11c6f6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:37:46.567074 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cede0e13-0b73-420d-95c5-01883b11c6f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:37:46.567540 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cede0e13-0b73-420d-95c5-01883b11c6f6 admin admin] PATH_INFO: `/v3/projects/90862fc01a7b48e38ba4ef60673c253e` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:37:46.568969 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cede0e13-0b73-420d-95c5-01883b11c6f6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=90862fc01a7b48e38ba4ef60673c253e)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:37:46.571157 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cede0e13-0b73-420d-95c5-01883b11c6f6 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:37:46.606840 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 961/1933] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 25 05:37:46 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:37:46.607618 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-174363ea-f025-44b0-bdc6-9e3fd55116fe None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:37:46.619040 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-174363ea-f025-44b0-bdc6-9e3fd55116fe None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:37:46.625514 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b34c8acb-bde6-442c-bbc8-90665f38accf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:37:46.626153 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34c8acb-bde6-442c-bbc8-90665f38accf admin admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:37:46.626410 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34c8acb-bde6-442c-bbc8-90665f38accf admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:37:46.626692 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34c8acb-bde6-442c-bbc8-90665f38accf admin admin] PATH_INFO: `/v3/roles` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:37:46.627454 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b34c8acb-bde6-442c-bbc8-90665f38accf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:37:46.629506 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b34c8acb-bde6-442c-bbc8-90665f38accf admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:37:46.643355 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-cede0e13-0b73-420d-95c5-01883b11c6f6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '90862fc01a7b48e38ba4ef60673c253e', 'request_id': 'req-cede0e13-0b73-420d-95c5-01883b11c6f6'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:37:46.644944 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 962/1934] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jul 25 05:37:46 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 05:37:46.646823 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b34c8acb-bde6-442c-bbc8-90665f38accf admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:37:46.649060 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-b34c8acb-bde6-442c-bbc8-90665f38accf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:37:46.649846 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-cede0e13-0b73-420d-95c5-01883b11c6f6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '90862fc01a7b48e38ba4ef60673c253e', 'request_id': 'req-cede0e13-0b73-420d-95c5-01883b11c6f6'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:37:46.657630 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-cede0e13-0b73-420d-95c5-01883b11c6f6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '90862fc01a7b48e38ba4ef60673c253e', 'request_id': 'req-cede0e13-0b73-420d-95c5-01883b11c6f6'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:37:46.676502 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-cede0e13-0b73-420d-95c5-01883b11c6f6 admin admin] The token cache is being invalidate because project 90862fc01a7b48e38ba4ef60673c253e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80249) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 05:37:46.677330 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-cede0e13-0b73-420d-95c5-01883b11c6f6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:37:46.678076 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-cede0e13-0b73-420d-95c5-01883b11c6f6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:37:46.679717 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 973/1935] 10.222.0.33 () {58 vars in 1232 bytes} [Mon Jul 25 05:37:46 2022] DELETE /identity/v3/projects/90862fc01a7b48e38ba4ef60673c253e => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 05:37:46.680651 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cede0e13-0b73-420d-95c5-01883b11c6f6 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:37:46.681996 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-cede0e13-0b73-420d-95c5-01883b11c6f6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:37:46.709564 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:37:46.727921 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b34c8acb-bde6-442c-bbc8-90665f38accf admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:37:46.746568 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:37:46.772151 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cede0e13-0b73-420d-95c5-01883b11c6f6 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:37:46.777753 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd49cd5d-935d-48e8-9407-bb396dbeb5dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:37:46.778752 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd49cd5d-935d-48e8-9407-bb396dbeb5dc admin admin] REQUEST_METHOD: `PUT` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:37:46.779111 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd49cd5d-935d-48e8-9407-bb396dbeb5dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:37:46.779461 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd49cd5d-935d-48e8-9407-bb396dbeb5dc admin admin] PATH_INFO: `/v3/projects/54430781b5024e5e92b8399f9bc37e86/users/6ff9830c0fcd469386cf9c014ac1798a/roles/5074e74e35f54d4b9bb415c34999a82a` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:37:46.793691 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94668081-65ba-4a88-a72f-8d30e87d25ac service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db57b478c25643dbaf936f90f6e17d0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92afba266cce4d1bad4134c351e74be3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:37:46.794408 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94668081-65ba-4a88-a72f-8d30e87d25ac service cinder] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:37:46.794770 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94668081-65ba-4a88-a72f-8d30e87d25ac service cinder] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:37:46.795056 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94668081-65ba-4a88-a72f-8d30e87d25ac service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:37:46.795933 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd49cd5d-935d-48e8-9407-bb396dbeb5dc admin admin] RBAC: Authorizing `identity:create_grant(project_id=54430781b5024e5e92b8399f9bc37e86, user_id=6ff9830c0fcd469386cf9c014ac1798a, role_id=5074e74e35f54d4b9bb415c34999a82a)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:37:46.797196 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-94668081-65ba-4a88-a72f-8d30e87d25ac service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:37:46.799319 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd49cd5d-935d-48e8-9407-bb396dbeb5dc admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:37:46.820870 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 974/1936] 10.222.0.33 () {60 vars in 1482 bytes} [Mon Jul 25 05:37:46 2022] PUT /identity/v3/projects/54430781b5024e5e92b8399f9bc37e86/users/6ff9830c0fcd469386cf9c014ac1798a/roles/5074e74e35f54d4b9bb415c34999a82a => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 05:37:46.822090 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd49cd5d-935d-48e8-9407-bb396dbeb5dc admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:37:46.826094 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-dd49cd5d-935d-48e8-9407-bb396dbeb5dc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:37:46.883017 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94668081-65ba-4a88-a72f-8d30e87d25ac service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:37:46.884466 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94668081-65ba-4a88-a72f-8d30e87d25ac service cinder] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:37:46.897571 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 963/1937] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jul 25 05:37:46 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 249 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 05:37:46.897571 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e3b901-c413-43e2-a391-973e278aef34 None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:37:46.897571 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e3b901-c413-43e2-a391-973e278aef34 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:37:46.897571 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e3b901-c413-43e2-a391-973e278aef34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:37:46.929383 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd49cd5d-935d-48e8-9407-bb396dbeb5dc admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:37:46.936481 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e1a6ee-440e-4bdf-a068-cdf39d3d58ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:37:46.937541 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e1a6ee-440e-4bdf-a068-cdf39d3d58ec admin admin] REQUEST_METHOD: `DELETE` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:37:46.937930 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e1a6ee-440e-4bdf-a068-cdf39d3d58ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:37:46.938493 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e1a6ee-440e-4bdf-a068-cdf39d3d58ec admin admin] PATH_INFO: `/v3/users/3ee52718ab8d4c7e910a1cb323334465` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:37:46.940015 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0e1a6ee-440e-4bdf-a068-cdf39d3d58ec admin admin] RBAC: Authorizing `identity:delete_user(user_id=3ee52718ab8d4c7e910a1cb323334465)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:37:46.942194 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0e1a6ee-440e-4bdf-a068-cdf39d3d58ec admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:37:46.969793 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-89e3b901-c413-43e2-a391-973e278aef34 None None] MFA Rules not processed for user `6ff9830c0fcd469386cf9c014ac1798a`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:37:46.973696 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-89e3b901-c413-43e2-a391-973e278aef34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:37:47.053527 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-d0e1a6ee-440e-4bdf-a068-cdf39d3d58ec admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3ee52718ab8d4c7e910a1cb323334465'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:37:47.055982 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-d0e1a6ee-440e-4bdf-a068-cdf39d3d58ec admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3ee52718ab8d4c7e910a1cb323334465'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:37:47.085764 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-d0e1a6ee-440e-4bdf-a068-cdf39d3d58ec admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3ee52718ab8d4c7e910a1cb323334465'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:37:47.120615 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 964/1938] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 25 05:37:46 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 225 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:37:47.125148 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e3b901-c413-43e2-a391-973e278aef34 None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:37:47.125997 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-d0e1a6ee-440e-4bdf-a068-cdf39d3d58ec admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3ee52718ab8d4c7e910a1cb323334465'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:37:47.127600 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-d0e1a6ee-440e-4bdf-a068-cdf39d3d58ec admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3ee52718ab8d4c7e910a1cb323334465'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:37:47.141743 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e3b901-c413-43e2-a391-973e278aef34 None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:37:47.155464 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2004bd0b-4ca5-4990-a0cf-42d20d82376d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:37:47.156830 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2004bd0b-4ca5-4990-a0cf-42d20d82376d admin admin] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:37:47.157150 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2004bd0b-4ca5-4990-a0cf-42d20d82376d admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:37:47.157635 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2004bd0b-4ca5-4990-a0cf-42d20d82376d admin admin] PATH_INFO: `/v3/projects` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:37:47.158379 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2004bd0b-4ca5-4990-a0cf-42d20d82376d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:37:47.162484 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2004bd0b-4ca5-4990-a0cf-42d20d82376d admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:37:47.170220 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 975/1939] 10.222.0.33 () {58 vars in 1223 bytes} [Mon Jul 25 05:37:46 2022] DELETE /identity/v3/users/3ee52718ab8d4c7e910a1cb323334465 => generated 0 bytes in 349 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 05:37:47.236344 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 965/1940] 10.222.0.33 () {60 vars in 1152 bytes} [Mon Jul 25 05:37:47 2022] POST /identity/v3/projects => generated 381 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 05:37:47.239398 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e1a6ee-440e-4bdf-a068-cdf39d3d58ec admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:37:47.244743 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-d0e1a6ee-440e-4bdf-a068-cdf39d3d58ec admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:37:47.250955 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2004bd0b-4ca5-4990-a0cf-42d20d82376d admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:37:47.271522 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-2004bd0b-4ca5-4990-a0cf-42d20d82376d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:37:47.333568 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:37:47.372959 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:37:47.407438 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e1a6ee-440e-4bdf-a068-cdf39d3d58ec admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:37:47.411891 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2004bd0b-4ca5-4990-a0cf-42d20d82376d admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:37:47.419556 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a9942b-48c3-4b7f-8ae9-94829d8ae540 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:37:47.420928 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a9942b-48c3-4b7f-8ae9-94829d8ae540 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:37:47.421316 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a9942b-48c3-4b7f-8ae9-94829d8ae540 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:37:47.421789 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f82dab8e-583e-42f0-b116-eb96e056c4b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:37:47.422032 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a9942b-48c3-4b7f-8ae9-94829d8ae540 admin admin] PATH_INFO: `/v3/projects/7dc9e81bf3f44ebca324dad8bb89b55e` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:37:47.423888 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43a9942b-48c3-4b7f-8ae9-94829d8ae540 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7dc9e81bf3f44ebca324dad8bb89b55e)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:37:47.424223 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82dab8e-583e-42f0-b116-eb96e056c4b9 admin admin] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:37:47.424461 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82dab8e-583e-42f0-b116-eb96e056c4b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:37:47.426589 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82dab8e-583e-42f0-b116-eb96e056c4b9 admin admin] PATH_INFO: `/v3/users` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:37:47.426846 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43a9942b-48c3-4b7f-8ae9-94829d8ae540 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:37:47.427402 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f82dab8e-583e-42f0-b116-eb96e056c4b9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:37:47.438020 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f82dab8e-583e-42f0-b116-eb96e056c4b9 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:37:47.481198 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 976/1941] 10.222.0.33 () {60 vars in 1143 bytes} [Mon Jul 25 05:37:47 2022] POST /identity/v3/users => generated 297 bytes in 242 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 25 05:37:47.486382 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f82dab8e-583e-42f0-b116-eb96e056c4b9 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:37:47.503838 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-43a9942b-48c3-4b7f-8ae9-94829d8ae540 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7dc9e81bf3f44ebca324dad8bb89b55e', 'request_id': 'req-43a9942b-48c3-4b7f-8ae9-94829d8ae540'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:37:47.511153 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-43a9942b-48c3-4b7f-8ae9-94829d8ae540 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7dc9e81bf3f44ebca324dad8bb89b55e', 'request_id': 'req-43a9942b-48c3-4b7f-8ae9-94829d8ae540'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:37:47.512273 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f82dab8e-583e-42f0-b116-eb96e056c4b9 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:37:47.518922 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-43a9942b-48c3-4b7f-8ae9-94829d8ae540 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7dc9e81bf3f44ebca324dad8bb89b55e', 'request_id': 'req-43a9942b-48c3-4b7f-8ae9-94829d8ae540'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:37:47.543481 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-43a9942b-48c3-4b7f-8ae9-94829d8ae540 admin admin] The token cache is being invalidate because project 7dc9e81bf3f44ebca324dad8bb89b55e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80248) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 05:37:47.544224 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-43a9942b-48c3-4b7f-8ae9-94829d8ae540 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:37:47.546128 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-43a9942b-48c3-4b7f-8ae9-94829d8ae540 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:37:47.548286 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 966/1942] 10.222.0.33 () {58 vars in 1232 bytes} [Mon Jul 25 05:37:47 2022] DELETE /identity/v3/projects/7dc9e81bf3f44ebca324dad8bb89b55e => generated 0 bytes in 304 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 05:37:47.552117 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a9942b-48c3-4b7f-8ae9-94829d8ae540 admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:37:47.554525 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-43a9942b-48c3-4b7f-8ae9-94829d8ae540 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:37:47.652358 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce13f3d6-d41f-4f3a-8ae9-0a93933b1ac3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:37:47.653334 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce13f3d6-d41f-4f3a-8ae9-0a93933b1ac3 admin admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:37:47.653772 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce13f3d6-d41f-4f3a-8ae9-0a93933b1ac3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:37:47.654173 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce13f3d6-d41f-4f3a-8ae9-0a93933b1ac3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:37:47.655059 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce13f3d6-d41f-4f3a-8ae9-0a93933b1ac3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:37:47.657957 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce13f3d6-d41f-4f3a-8ae9-0a93933b1ac3 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:37:47.701133 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 977/1943] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jul 25 05:37:47 2022] GET /identity/v3/roles => generated 1420 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 25 05:37:47.702274 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce13f3d6-d41f-4f3a-8ae9-0a93933b1ac3 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:37:47.706848 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-ce13f3d6-d41f-4f3a-8ae9-0a93933b1ac3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:37:47.745562 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce13f3d6-d41f-4f3a-8ae9-0a93933b1ac3 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:37:47.758201 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c6c06b-27cc-4bbf-a26c-755472998125 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:37:47.759239 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c6c06b-27cc-4bbf-a26c-755472998125 admin admin] REQUEST_METHOD: `PUT` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:37:47.759412 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c6c06b-27cc-4bbf-a26c-755472998125 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:37:47.759692 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c6c06b-27cc-4bbf-a26c-755472998125 admin admin] PATH_INFO: `/v3/projects/f504125d6ec1415aa6e75f0a53480ed7/users/79e4d967bd3f434cb4e0e54fb0bbdbe5/roles/39e5c666285a42fe8fc2c3afca6260d1` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:37:47.767683 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a9942b-48c3-4b7f-8ae9-94829d8ae540 admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:37:47.784384 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4922bef2-2d14-4e15-a7a3-8b8f2dadf9fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:37:47.785495 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4922bef2-2d14-4e15-a7a3-8b8f2dadf9fa admin admin] REQUEST_METHOD: `DELETE` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:37:47.792865 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4922bef2-2d14-4e15-a7a3-8b8f2dadf9fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:37:47.802331 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82c6c06b-27cc-4bbf-a26c-755472998125 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f504125d6ec1415aa6e75f0a53480ed7, user_id=79e4d967bd3f434cb4e0e54fb0bbdbe5, role_id=39e5c666285a42fe8fc2c3afca6260d1)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:37:47.809557 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82c6c06b-27cc-4bbf-a26c-755472998125 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:37:47.814782 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4922bef2-2d14-4e15-a7a3-8b8f2dadf9fa admin admin] PATH_INFO: `/v3/users/3e3a518b2cf54bf2b7dc8e84295782c2` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:37:47.819005 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4922bef2-2d14-4e15-a7a3-8b8f2dadf9fa admin admin] RBAC: Authorizing `identity:delete_user(user_id=3e3a518b2cf54bf2b7dc8e84295782c2)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:37:47.821418 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4922bef2-2d14-4e15-a7a3-8b8f2dadf9fa admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:37:47.826816 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 978/1944] 10.222.0.33 () {60 vars in 1482 bytes} [Mon Jul 25 05:37:47 2022] PUT /identity/v3/projects/f504125d6ec1415aa6e75f0a53480ed7/users/79e4d967bd3f434cb4e0e54fb0bbdbe5/roles/39e5c666285a42fe8fc2c3afca6260d1 => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 05:37:47.832081 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c6c06b-27cc-4bbf-a26c-755472998125 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:37:47.975056 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-4922bef2-2d14-4e15-a7a3-8b8f2dadf9fa admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3e3a518b2cf54bf2b7dc8e84295782c2'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:37:47.983000 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:37:48.020046 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-4922bef2-2d14-4e15-a7a3-8b8f2dadf9fa admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3e3a518b2cf54bf2b7dc8e84295782c2'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:37:48.020574 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-4922bef2-2d14-4e15-a7a3-8b8f2dadf9fa admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3e3a518b2cf54bf2b7dc8e84295782c2'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:37:48.024384 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c6c06b-27cc-4bbf-a26c-755472998125 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:37:48.034209 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac61c7af-22ba-467c-9272-2495e5ca77f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:37:48.035321 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac61c7af-22ba-467c-9272-2495e5ca77f4 admin admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:37:48.035744 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac61c7af-22ba-467c-9272-2495e5ca77f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:37:48.036159 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac61c7af-22ba-467c-9272-2495e5ca77f4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:37:48.037149 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac61c7af-22ba-467c-9272-2495e5ca77f4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:37:48.039118 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac61c7af-22ba-467c-9272-2495e5ca77f4 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:37:48.043740 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-4922bef2-2d14-4e15-a7a3-8b8f2dadf9fa admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3e3a518b2cf54bf2b7dc8e84295782c2'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:37:48.044954 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-4922bef2-2d14-4e15-a7a3-8b8f2dadf9fa admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3e3a518b2cf54bf2b7dc8e84295782c2'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:37:48.067998 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 967/1945] 10.222.0.33 () {58 vars in 1223 bytes} [Mon Jul 25 05:37:47 2022] DELETE /identity/v3/users/3e3a518b2cf54bf2b7dc8e84295782c2 => generated 0 bytes in 516 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 25 05:37:48.070921 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2f85bb-d964-4176-959b-78658a7ca941 None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:37:48.070921 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2f85bb-d964-4176-959b-78658a7ca941 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:37:48.070921 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2f85bb-d964-4176-959b-78658a7ca941 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:37:48.105658 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-7a2f85bb-d964-4176-959b-78658a7ca941 None None] MFA Rules not processed for user `357148a4c5564dc3ab9b2273513ec470`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:37:48.107007 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-7a2f85bb-d964-4176-959b-78658a7ca941 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:37:48.127454 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 979/1946] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jul 25 05:37:47 2022] GET /identity/v3/roles => generated 1420 bytes in 295 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 05:37:48.134113 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac61c7af-22ba-467c-9272-2495e5ca77f4 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:37:48.156574 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-ac61c7af-22ba-467c-9272-2495e5ca77f4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:37:48.213325 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac61c7af-22ba-467c-9272-2495e5ca77f4 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:37:48.293701 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:37:48.314110 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc3b4d6d-dab8-48d6-bbcd-650078a46872 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:37:48.315711 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3b4d6d-dab8-48d6-bbcd-650078a46872 admin admin] REQUEST_METHOD: `PUT` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:37:48.315829 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3b4d6d-dab8-48d6-bbcd-650078a46872 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:37:48.316073 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3b4d6d-dab8-48d6-bbcd-650078a46872 admin admin] PATH_INFO: `/v3/projects/f504125d6ec1415aa6e75f0a53480ed7/users/79e4d967bd3f434cb4e0e54fb0bbdbe5/roles/39e5c666285a42fe8fc2c3afca6260d1` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:37:48.319184 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc3b4d6d-dab8-48d6-bbcd-650078a46872 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f504125d6ec1415aa6e75f0a53480ed7, user_id=79e4d967bd3f434cb4e0e54fb0bbdbe5, role_id=39e5c666285a42fe8fc2c3afca6260d1)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:37:48.320211 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 968/1947] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 25 05:37:48 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 251 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:37:48.324745 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc3b4d6d-dab8-48d6-bbcd-650078a46872 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:37:48.330929 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a2f85bb-d964-4176-959b-78658a7ca941 None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:37:48.333065 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-7a2f85bb-d964-4176-959b-78658a7ca941 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:37:48.350560 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 980/1948] 10.222.0.33 () {60 vars in 1482 bytes} [Mon Jul 25 05:37:48 2022] PUT /identity/v3/projects/f504125d6ec1415aa6e75f0a53480ed7/users/79e4d967bd3f434cb4e0e54fb0bbdbe5/roles/39e5c666285a42fe8fc2c3afca6260d1 => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 05:37:48.353269 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc1b462-2a48-4a1a-9be4-bec6205f4c08 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:37:48.354029 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc1b462-2a48-4a1a-9be4-bec6205f4c08 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:37:48.355381 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc1b462-2a48-4a1a-9be4-bec6205f4c08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:37:48.424465 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-6cc1b462-2a48-4a1a-9be4-bec6205f4c08 None None] MFA Rules not processed for user `79e4d967bd3f434cb4e0e54fb0bbdbe5`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:37:48.425573 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-6cc1b462-2a48-4a1a-9be4-bec6205f4c08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:37:48.447372 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a2f85bb-d964-4176-959b-78658a7ca941 None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:37:48.576532 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b1308a3-c5fa-431b-82a8-e146a748a088 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db57b478c25643dbaf936f90f6e17d0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92afba266cce4d1bad4134c351e74be3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:37:48.577429 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1308a3-c5fa-431b-82a8-e146a748a088 service cinder] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:37:48.577864 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1308a3-c5fa-431b-82a8-e146a748a088 service cinder] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:37:48.578363 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1308a3-c5fa-431b-82a8-e146a748a088 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:37:48.582322 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-5b1308a3-c5fa-431b-82a8-e146a748a088 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:37:48.595970 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 981/1949] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 25 05:37:48 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 241 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:37:48.595970 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc1b462-2a48-4a1a-9be4-bec6205f4c08 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:37:48.595970 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-6cc1b462-2a48-4a1a-9be4-bec6205f4c08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:37:48.652281 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:DELETE=3,keystone:SELECT=96,keystone:INSERT=1 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:37:48.714943 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc1b462-2a48-4a1a-9be4-bec6205f4c08 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:37:48.716102 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d03fa922-b1b8-42ac-83d7-7dd5784e6e38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:37:48.717048 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03fa922-b1b8-42ac-83d7-7dd5784e6e38 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:37:48.717362 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03fa922-b1b8-42ac-83d7-7dd5784e6e38 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:37:48.717646 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03fa922-b1b8-42ac-83d7-7dd5784e6e38 admin admin] PATH_INFO: `/v3/projects/93a42efeecef4a6e9da5a5d09dc5db9e` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:37:48.718860 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d03fa922-b1b8-42ac-83d7-7dd5784e6e38 admin admin] RBAC: Authorizing `identity:delete_project(project_id=93a42efeecef4a6e9da5a5d09dc5db9e)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:37:48.721353 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d03fa922-b1b8-42ac-83d7-7dd5784e6e38 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:37:48.732102 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b1308a3-c5fa-431b-82a8-e146a748a088 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:37:48.733691 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b1308a3-c5fa-431b-82a8-e146a748a088 service cinder] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:37:48.743913 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 969/1950] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jul 25 05:37:48 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 413 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 05:37:48.753411 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b1308a3-c5fa-431b-82a8-e146a748a088 service cinder] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:37:48.755666 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-5b1308a3-c5fa-431b-82a8-e146a748a088 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:37:48.790828 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b1308a3-c5fa-431b-82a8-e146a748a088 service cinder] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:37:48.809145 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-d03fa922-b1b8-42ac-83d7-7dd5784e6e38 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '93a42efeecef4a6e9da5a5d09dc5db9e', 'request_id': 'req-d03fa922-b1b8-42ac-83d7-7dd5784e6e38'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:37:48.815181 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-d03fa922-b1b8-42ac-83d7-7dd5784e6e38 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '93a42efeecef4a6e9da5a5d09dc5db9e', 'request_id': 'req-d03fa922-b1b8-42ac-83d7-7dd5784e6e38'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:37:48.834113 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-d03fa922-b1b8-42ac-83d7-7dd5784e6e38 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '93a42efeecef4a6e9da5a5d09dc5db9e', 'request_id': 'req-d03fa922-b1b8-42ac-83d7-7dd5784e6e38'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:37:48.859481 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:37:48.865854 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-d03fa922-b1b8-42ac-83d7-7dd5784e6e38 admin admin] The token cache is being invalidate because project 93a42efeecef4a6e9da5a5d09dc5db9e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80249) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 05:37:48.866401 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-d03fa922-b1b8-42ac-83d7-7dd5784e6e38 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:37:48.867552 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-d03fa922-b1b8-42ac-83d7-7dd5784e6e38 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:37:48.871111 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 982/1951] 10.222.0.33 () {58 vars in 1232 bytes} [Mon Jul 25 05:37:48 2022] DELETE /identity/v3/projects/93a42efeecef4a6e9da5a5d09dc5db9e => generated 0 bytes in 277 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 05:37:48.872891 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d03fa922-b1b8-42ac-83d7-7dd5784e6e38 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:37:48.875170 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-d03fa922-b1b8-42ac-83d7-7dd5784e6e38 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:37:48.926102 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d751dc-b9e4-4f57-9f77-83bf980ba311 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db57b478c25643dbaf936f90f6e17d0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92afba266cce4d1bad4134c351e74be3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:37:48.926728 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d751dc-b9e4-4f57-9f77-83bf980ba311 service cinder] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:37:48.927230 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d751dc-b9e4-4f57-9f77-83bf980ba311 service cinder] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:37:48.927550 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d751dc-b9e4-4f57-9f77-83bf980ba311 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:37:48.929826 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-89d751dc-b9e4-4f57-9f77-83bf980ba311 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:37:49.014249 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89d751dc-b9e4-4f57-9f77-83bf980ba311 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:37:49.016215 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89d751dc-b9e4-4f57-9f77-83bf980ba311 service cinder] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:37:49.031482 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 970/1952] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jul 25 05:37:48 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 287 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 05:37:49.033363 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6d81de-3d7e-401c-bb66-42d793b0a5d2 None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:37:49.033363 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6d81de-3d7e-401c-bb66-42d793b0a5d2 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:37:49.033552 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6d81de-3d7e-401c-bb66-42d793b0a5d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:37:49.036127 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d03fa922-b1b8-42ac-83d7-7dd5784e6e38 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:37:49.064828 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e55023-8714-43b8-9710-697bee235656 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:37:49.064962 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-4f6d81de-3d7e-401c-bb66-42d793b0a5d2 None None] MFA Rules not processed for user `1eae540242d94c10b0ad33960a855f08`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:37:49.065058 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e55023-8714-43b8-9710-697bee235656 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:37:49.065058 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e55023-8714-43b8-9710-697bee235656 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:37:49.065058 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e55023-8714-43b8-9710-697bee235656 admin admin] PATH_INFO: `/v3/projects/d799bb4c8a1d45f8874976be48c31b3a` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:37:49.065291 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-4f6d81de-3d7e-401c-bb66-42d793b0a5d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:37:49.065377 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5e55023-8714-43b8-9710-697bee235656 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d799bb4c8a1d45f8874976be48c31b3a)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:37:49.066775 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5e55023-8714-43b8-9710-697bee235656 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:37:49.089002 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 971/1953] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 25 05:37:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:37:49.093744 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e837dc0-1bf7-4fee-83ff-958bed9638ef None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:37:49.094057 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e837dc0-1bf7-4fee-83ff-958bed9638ef None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:37:49.094348 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e837dc0-1bf7-4fee-83ff-958bed9638ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:37:49.127761 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-3e837dc0-1bf7-4fee-83ff-958bed9638ef None None] MFA Rules not processed for user `1eae540242d94c10b0ad33960a855f08`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:37:49.128085 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-e5e55023-8714-43b8-9710-697bee235656 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd799bb4c8a1d45f8874976be48c31b3a', 'request_id': 'req-e5e55023-8714-43b8-9710-697bee235656'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:37:49.129354 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-3e837dc0-1bf7-4fee-83ff-958bed9638ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:37:49.135689 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-e5e55023-8714-43b8-9710-697bee235656 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd799bb4c8a1d45f8874976be48c31b3a', 'request_id': 'req-e5e55023-8714-43b8-9710-697bee235656'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:37:49.143669 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-e5e55023-8714-43b8-9710-697bee235656 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd799bb4c8a1d45f8874976be48c31b3a', 'request_id': 'req-e5e55023-8714-43b8-9710-697bee235656'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:37:49.165990 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-e5e55023-8714-43b8-9710-697bee235656 admin admin] The token cache is being invalidate because project d799bb4c8a1d45f8874976be48c31b3a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80249) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 05:37:49.166393 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-e5e55023-8714-43b8-9710-697bee235656 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:37:49.168023 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-e5e55023-8714-43b8-9710-697bee235656 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:37:49.173989 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 983/1954] 10.222.0.33 () {58 vars in 1232 bytes} [Mon Jul 25 05:37:48 2022] DELETE /identity/v3/projects/d799bb4c8a1d45f8874976be48c31b3a => generated 0 bytes in 302 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 05:37:49.183618 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a6a9a3-2ad6-4688-8a7d-1aabfc5f069e None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:37:49.184222 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a6a9a3-2ad6-4688-8a7d-1aabfc5f069e None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:37:49.184762 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a6a9a3-2ad6-4688-8a7d-1aabfc5f069e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:37:49.212468 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-34a6a9a3-2ad6-4688-8a7d-1aabfc5f069e None None] MFA Rules not processed for user `1eae540242d94c10b0ad33960a855f08`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:37:49.214759 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-34a6a9a3-2ad6-4688-8a7d-1aabfc5f069e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:37:49.257201 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 972/1955] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 25 05:37:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 05:37:49.258334 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e837dc0-1bf7-4fee-83ff-958bed9638ef None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:37:49.279300 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e837dc0-1bf7-4fee-83ff-958bed9638ef None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:37:49.292334 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f49a58-72bd-402c-9a83-99befb9fcdb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:37:49.293112 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f49a58-72bd-402c-9a83-99befb9fcdb9 admin admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:37:49.293536 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f49a58-72bd-402c-9a83-99befb9fcdb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:37:49.293824 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f49a58-72bd-402c-9a83-99befb9fcdb9 admin admin] PATH_INFO: `/v3/domains` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:37:49.296757 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6f49a58-72bd-402c-9a83-99befb9fcdb9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:37:49.296974 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6f49a58-72bd-402c-9a83-99befb9fcdb9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:37:49.303323 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6f49a58-72bd-402c-9a83-99befb9fcdb9 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:37:49.308134 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:37:49.344913 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 973/1956] 10.222.0.33 () {58 vars in 1152 bytes} [Mon Jul 25 05:37:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:37:49.345824 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f49a58-72bd-402c-9a83-99befb9fcdb9 admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:37:49.366146 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f49a58-72bd-402c-9a83-99befb9fcdb9 admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:37:49.381764 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-024c9cf0-e914-4357-af00-3f28332976f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:37:49.381764 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024c9cf0-e914-4357-af00-3f28332976f6 admin admin] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:37:49.381764 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024c9cf0-e914-4357-af00-3f28332976f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:37:49.381764 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024c9cf0-e914-4357-af00-3f28332976f6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:37:49.382322 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-024c9cf0-e914-4357-af00-3f28332976f6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:37:49.382322 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-024c9cf0-e914-4357-af00-3f28332976f6 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:37:49.382663 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 984/1957] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 25 05:37:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 05:37:49.384020 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a6a9a3-2ad6-4688-8a7d-1aabfc5f069e None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:37:49.385741 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-34a6a9a3-2ad6-4688-8a7d-1aabfc5f069e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:37:49.412120 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 974/1958] 10.222.0.33 () {60 vars in 1152 bytes} [Mon Jul 25 05:37:49 2022] POST /identity/v3/projects => generated 387 bytes in 67 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 05:37:49.413606 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e1ace3-8088-4b93-9e61-97b0a342a74b None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:37:49.414068 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e1ace3-8088-4b93-9e61-97b0a342a74b None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:37:49.414488 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e1ace3-8088-4b93-9e61-97b0a342a74b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:37:49.424322 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:37:49.441207 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-19e1ace3-8088-4b93-9e61-97b0a342a74b None None] MFA Rules not processed for user `1eae540242d94c10b0ad33960a855f08`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:37:49.442565 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-19e1ace3-8088-4b93-9e61-97b0a342a74b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:37:49.459575 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a6a9a3-2ad6-4688-8a7d-1aabfc5f069e None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:37:49.504060 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 975/1959] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 25 05:37:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:37:49.504700 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e1ace3-8088-4b93-9e61-97b0a342a74b None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:37:49.514140 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f23b8e6-e5ee-40c2-8734-da75da4364b8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db57b478c25643dbaf936f90f6e17d0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92afba266cce4d1bad4134c351e74be3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:37:49.514773 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f23b8e6-e5ee-40c2-8734-da75da4364b8 service cinder] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:37:49.514936 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e1ace3-8088-4b93-9e61-97b0a342a74b None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:37:49.515051 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f23b8e6-e5ee-40c2-8734-da75da4364b8 service cinder] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:37:49.515346 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f23b8e6-e5ee-40c2-8734-da75da4364b8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:37:49.517193 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-1f23b8e6-e5ee-40c2-8734-da75da4364b8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:37:49.520640 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7064124d-b759-4c2c-8049-7c67bf7385a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:37:49.521344 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7064124d-b759-4c2c-8049-7c67bf7385a4 admin admin] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:37:49.521637 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7064124d-b759-4c2c-8049-7c67bf7385a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:37:49.521935 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7064124d-b759-4c2c-8049-7c67bf7385a4 admin admin] PATH_INFO: `/v3/users` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:37:49.522802 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7064124d-b759-4c2c-8049-7c67bf7385a4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:37:49.524734 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7064124d-b759-4c2c-8049-7c67bf7385a4 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:37:49.549269 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 976/1960] 10.222.0.33 () {60 vars in 1143 bytes} [Mon Jul 25 05:37:49 2022] POST /identity/v3/users => generated 294 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 05:37:49.550125 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7064124d-b759-4c2c-8049-7c67bf7385a4 admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:37:49.552200 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-7064124d-b759-4c2c-8049-7c67bf7385a4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:37:49.568184 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7064124d-b759-4c2c-8049-7c67bf7385a4 admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:37:49.573126 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e1877e-0e5b-4a29-a89b-685080756e13 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db57b478c25643dbaf936f90f6e17d0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92afba266cce4d1bad4134c351e74be3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:37:49.574191 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e1877e-0e5b-4a29-a89b-685080756e13 service cinder] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:37:49.574632 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e1877e-0e5b-4a29-a89b-685080756e13 service cinder] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:37:49.575038 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e1877e-0e5b-4a29-a89b-685080756e13 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:37:49.577567 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-c0e1877e-0e5b-4a29-a89b-685080756e13 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:37:49.590064 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f23b8e6-e5ee-40c2-8734-da75da4364b8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:37:49.591888 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f23b8e6-e5ee-40c2-8734-da75da4364b8 service cinder] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:37:49.604770 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 985/1961] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jul 25 05:37:49 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 221 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 05:37:49.605790 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f23b8e6-e5ee-40c2-8734-da75da4364b8 service cinder] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:37:49.616947 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f23b8e6-e5ee-40c2-8734-da75da4364b8 service cinder] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:37:49.623444 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4acecef6-1d6a-4d04-bf61-5c9bc4eb1cbf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:37:49.624135 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4acecef6-1d6a-4d04-bf61-5c9bc4eb1cbf admin admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:37:49.624425 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4acecef6-1d6a-4d04-bf61-5c9bc4eb1cbf admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:37:49.624733 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4acecef6-1d6a-4d04-bf61-5c9bc4eb1cbf admin admin] PATH_INFO: `/v3/domains` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:37:49.625576 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4acecef6-1d6a-4d04-bf61-5c9bc4eb1cbf admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:37:49.625990 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4acecef6-1d6a-4d04-bf61-5c9bc4eb1cbf admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:37:49.627617 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4acecef6-1d6a-4d04-bf61-5c9bc4eb1cbf admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:37:49.644532 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 986/1962] 10.222.0.33 () {58 vars in 1152 bytes} [Mon Jul 25 05:37:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:37:49.649009 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4acecef6-1d6a-4d04-bf61-5c9bc4eb1cbf admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:37:49.661668 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0e1877e-0e5b-4a29-a89b-685080756e13 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:37:49.661995 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4acecef6-1d6a-4d04-bf61-5c9bc4eb1cbf admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:37:49.663641 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0e1877e-0e5b-4a29-a89b-685080756e13 service cinder] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:37:49.668810 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f2f072-50b0-4aa7-a6ed-7c467a90a6e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:37:49.669601 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f2f072-50b0-4aa7-a6ed-7c467a90a6e9 admin admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:37:49.669894 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f2f072-50b0-4aa7-a6ed-7c467a90a6e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:37:49.670206 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f2f072-50b0-4aa7-a6ed-7c467a90a6e9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:37:49.671021 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7f2f072-50b0-4aa7-a6ed-7c467a90a6e9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:37:49.673868 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7f2f072-50b0-4aa7-a6ed-7c467a90a6e9 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:37:49.675955 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 977/1963] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jul 25 05:37:49 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 05:37:49.676847 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e1877e-0e5b-4a29-a89b-685080756e13 service cinder] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:37:49.690121 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e1877e-0e5b-4a29-a89b-685080756e13 service cinder] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:37:49.690531 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 987/1964] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jul 25 05:37:49 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 05:37:49.691630 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f2f072-50b0-4aa7-a6ed-7c467a90a6e9 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:37:49.696238 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aeede8c-35f9-4113-a76c-372f362ba6da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:37:49.697027 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aeede8c-35f9-4113-a76c-372f362ba6da admin admin] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:37:49.697315 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aeede8c-35f9-4113-a76c-372f362ba6da admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:37:49.697588 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aeede8c-35f9-4113-a76c-372f362ba6da admin admin] PATH_INFO: `/v3/projects` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:37:49.698409 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aeede8c-35f9-4113-a76c-372f362ba6da admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:37:49.700304 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aeede8c-35f9-4113-a76c-372f362ba6da admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:37:49.707516 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f2f072-50b0-4aa7-a6ed-7c467a90a6e9 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:37:49.715743 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69707030-739b-4f6d-8892-bea630b2af06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:37:49.716890 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69707030-739b-4f6d-8892-bea630b2af06 admin admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:37:49.717305 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69707030-739b-4f6d-8892-bea630b2af06 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:37:49.717691 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69707030-739b-4f6d-8892-bea630b2af06 admin admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:37:49.718567 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69707030-739b-4f6d-8892-bea630b2af06 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:37:49.720424 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69707030-739b-4f6d-8892-bea630b2af06 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:37:49.740817 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 988/1965] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jul 25 05:37:49 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 05:37:49.741983 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69707030-739b-4f6d-8892-bea630b2af06 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:37:49.750865 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 978/1966] 10.222.0.33 () {60 vars in 1152 bytes} [Mon Jul 25 05:37:49 2022] POST /identity/v3/projects => generated 375 bytes in 74 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 05:37:49.754814 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aeede8c-35f9-4113-a76c-372f362ba6da admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:37:49.758660 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69707030-739b-4f6d-8892-bea630b2af06 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:37:49.853586 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:37:49.888330 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68bec569-2846-4550-b96c-da717532f336 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:37:49.889221 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68bec569-2846-4550-b96c-da717532f336 admin admin] REQUEST_METHOD: `PUT` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:37:49.889549 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68bec569-2846-4550-b96c-da717532f336 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:37:49.889794 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68bec569-2846-4550-b96c-da717532f336 admin admin] PATH_INFO: `/v3/projects/c9bf4bcb38f74a488f2074c00874a56f/users/4d29be9a17894bb4b18f5f7692a1f8aa/roles/5074e74e35f54d4b9bb415c34999a82a` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:37:49.899413 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aeede8c-35f9-4113-a76c-372f362ba6da admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:37:49.907157 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc106344-3445-4a34-9f5a-fe0aefcfb2e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:37:49.907941 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc106344-3445-4a34-9f5a-fe0aefcfb2e6 admin admin] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:37:49.908847 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc106344-3445-4a34-9f5a-fe0aefcfb2e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:37:49.909146 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc106344-3445-4a34-9f5a-fe0aefcfb2e6 admin admin] PATH_INFO: `/v3/users` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:37:49.909890 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc106344-3445-4a34-9f5a-fe0aefcfb2e6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:37:49.912649 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc106344-3445-4a34-9f5a-fe0aefcfb2e6 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:37:49.919644 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:37:49.938634 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68bec569-2846-4550-b96c-da717532f336 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c9bf4bcb38f74a488f2074c00874a56f, user_id=4d29be9a17894bb4b18f5f7692a1f8aa, role_id=5074e74e35f54d4b9bb415c34999a82a)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:37:49.941422 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68bec569-2846-4550-b96c-da717532f336 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:37:49.950663 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 979/1967] 10.222.0.33 () {60 vars in 1143 bytes} [Mon Jul 25 05:37:49 2022] POST /identity/v3/users => generated 288 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 05:37:49.954722 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc106344-3445-4a34-9f5a-fe0aefcfb2e6 admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:37:49.992949 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 989/1968] 10.222.0.33 () {60 vars in 1482 bytes} [Mon Jul 25 05:37:49 2022] PUT /identity/v3/projects/c9bf4bcb38f74a488f2074c00874a56f/users/4d29be9a17894bb4b18f5f7692a1f8aa/roles/5074e74e35f54d4b9bb415c34999a82a => generated 0 bytes in 243 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 05:37:49.996080 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1330177-54a7-4f3f-a913-fc4400f93bfb None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:37:49.996389 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1330177-54a7-4f3f-a913-fc4400f93bfb None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:37:49.996694 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1330177-54a7-4f3f-a913-fc4400f93bfb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:37:49.997729 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc106344-3445-4a34-9f5a-fe0aefcfb2e6 admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:37:50.088142 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-b1330177-54a7-4f3f-a913-fc4400f93bfb None None] MFA Rules not processed for user `4d29be9a17894bb4b18f5f7692a1f8aa`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:37:50.090520 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-b1330177-54a7-4f3f-a913-fc4400f93bfb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:37:50.100666 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe50f2e-8451-41af-9598-880d7f4dcff4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:37:50.102757 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe50f2e-8451-41af-9598-880d7f4dcff4 admin admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:37:50.103483 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe50f2e-8451-41af-9598-880d7f4dcff4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:37:50.104993 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe50f2e-8451-41af-9598-880d7f4dcff4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:37:50.106023 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efe50f2e-8451-41af-9598-880d7f4dcff4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:37:50.107913 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efe50f2e-8451-41af-9598-880d7f4dcff4 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:37:50.159015 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 980/1969] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jul 25 05:37:49 2022] GET /identity/v3/roles => generated 1420 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 05:37:50.165654 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe50f2e-8451-41af-9598-880d7f4dcff4 admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:37:50.202808 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe50f2e-8451-41af-9598-880d7f4dcff4 admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:37:50.215152 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df059248-e079-409c-8ef5-07ff0315dd33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:37:50.216256 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df059248-e079-409c-8ef5-07ff0315dd33 admin admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:37:50.216730 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df059248-e079-409c-8ef5-07ff0315dd33 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:37:50.217560 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df059248-e079-409c-8ef5-07ff0315dd33 admin admin] PATH_INFO: `/v3/roles` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:37:50.218953 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df059248-e079-409c-8ef5-07ff0315dd33 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:37:50.222093 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df059248-e079-409c-8ef5-07ff0315dd33 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:37:50.238891 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 981/1970] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jul 25 05:37:50 2022] GET /identity/v3/roles => generated 1420 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 05:37:50.243975 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df059248-e079-409c-8ef5-07ff0315dd33 admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:37:50.251286 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 990/1971] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 25 05:37:49 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 257 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 05:37:50.256768 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1330177-54a7-4f3f-a913-fc4400f93bfb None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:37:50.261342 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df059248-e079-409c-8ef5-07ff0315dd33 admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:37:50.267558 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a2ab8e-c7f7-47bd-82c1-5a9f3cffe3e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:37:50.269379 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a2ab8e-c7f7-47bd-82c1-5a9f3cffe3e8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:37:50.269379 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a2ab8e-c7f7-47bd-82c1-5a9f3cffe3e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:37:50.269379 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a2ab8e-c7f7-47bd-82c1-5a9f3cffe3e8 admin admin] PATH_INFO: `/v3/projects/ae670cedc6c149a7bedea817cb280406/users/ae21aa22a8e04173a9a5e7500a390be8/roles/5074e74e35f54d4b9bb415c34999a82a` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:37:50.269651 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1330177-54a7-4f3f-a913-fc4400f93bfb None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:37:50.275675 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d06254-770a-44cf-bca7-b0fe76d9ec35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:37:50.276751 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d06254-770a-44cf-bca7-b0fe76d9ec35 admin admin] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:37:50.277179 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d06254-770a-44cf-bca7-b0fe76d9ec35 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:37:50.277460 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d06254-770a-44cf-bca7-b0fe76d9ec35 admin admin] PATH_INFO: `/v3/projects` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:37:50.278577 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63d06254-770a-44cf-bca7-b0fe76d9ec35 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:37:50.280557 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63d06254-770a-44cf-bca7-b0fe76d9ec35 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:37:50.286283 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5a2ab8e-c7f7-47bd-82c1-5a9f3cffe3e8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ae670cedc6c149a7bedea817cb280406, user_id=ae21aa22a8e04173a9a5e7500a390be8, role_id=5074e74e35f54d4b9bb415c34999a82a)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:37:50.289531 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5a2ab8e-c7f7-47bd-82c1-5a9f3cffe3e8 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:37:50.304916 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 982/1972] 10.222.0.33 () {60 vars in 1482 bytes} [Mon Jul 25 05:37:50 2022] PUT /identity/v3/projects/ae670cedc6c149a7bedea817cb280406/users/ae21aa22a8e04173a9a5e7500a390be8/roles/5074e74e35f54d4b9bb415c34999a82a => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 05:37:50.310896 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 991/1973] 10.222.0.33 () {60 vars in 1152 bytes} [Mon Jul 25 05:37:50 2022] POST /identity/v3/projects => generated 387 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 05:37:50.314820 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a2ab8e-c7f7-47bd-82c1-5a9f3cffe3e8 admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:37:50.318381 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e04baa-b1f5-42b7-aaa2-9b64bc40365d None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:37:50.318678 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e04baa-b1f5-42b7-aaa2-9b64bc40365d None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:37:50.318966 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e04baa-b1f5-42b7-aaa2-9b64bc40365d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:37:50.346529 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-67e04baa-b1f5-42b7-aaa2-9b64bc40365d None None] MFA Rules not processed for user `ae21aa22a8e04173a9a5e7500a390be8`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:37:50.348219 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-67e04baa-b1f5-42b7-aaa2-9b64bc40365d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:37:50.384063 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a2ab8e-c7f7-47bd-82c1-5a9f3cffe3e8 admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:37:50.432359 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:37:50.436317 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 992/1974] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 25 05:37:50 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:37:50.437513 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e04baa-b1f5-42b7-aaa2-9b64bc40365d None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:37:50.477762 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-627ea8c7-4eb5-4d6f-8b91-3a1a5fbc6167 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:37:50.478336 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627ea8c7-4eb5-4d6f-8b91-3a1a5fbc6167 admin admin] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:37:50.478627 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627ea8c7-4eb5-4d6f-8b91-3a1a5fbc6167 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:37:50.478894 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627ea8c7-4eb5-4d6f-8b91-3a1a5fbc6167 admin admin] PATH_INFO: `/v3/users` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:37:50.479636 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-627ea8c7-4eb5-4d6f-8b91-3a1a5fbc6167 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:37:50.481640 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-627ea8c7-4eb5-4d6f-8b91-3a1a5fbc6167 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:37:50.492717 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:37:50.503833 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e04baa-b1f5-42b7-aaa2-9b64bc40365d None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:37:50.504251 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 983/1975] 10.222.0.33 () {60 vars in 1143 bytes} [Mon Jul 25 05:37:50 2022] POST /identity/v3/users => generated 294 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 05:37:50.505278 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-627ea8c7-4eb5-4d6f-8b91-3a1a5fbc6167 admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:37:50.509512 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ebd352c-2bd6-440a-bdfc-bad19d73a6d8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db57b478c25643dbaf936f90f6e17d0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92afba266cce4d1bad4134c351e74be3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:37:50.510264 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ebd352c-2bd6-440a-bdfc-bad19d73a6d8 service cinder] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:37:50.511152 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ebd352c-2bd6-440a-bdfc-bad19d73a6d8 service cinder] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:37:50.511495 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ebd352c-2bd6-440a-bdfc-bad19d73a6d8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:37:50.513888 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-2ebd352c-2bd6-440a-bdfc-bad19d73a6d8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:37:50.521981 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-627ea8c7-4eb5-4d6f-8b91-3a1a5fbc6167 admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:37:50.528879 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2754989-59b5-43ef-86fe-065bc2c19d34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:37:50.529547 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2754989-59b5-43ef-86fe-065bc2c19d34 admin admin] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:37:50.529841 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2754989-59b5-43ef-86fe-065bc2c19d34 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:37:50.530128 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2754989-59b5-43ef-86fe-065bc2c19d34 admin admin] PATH_INFO: `/v3/projects` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:37:50.530835 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2754989-59b5-43ef-86fe-065bc2c19d34 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:37:50.533056 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2754989-59b5-43ef-86fe-065bc2c19d34 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:37:50.563397 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 984/1976] 10.222.0.33 () {60 vars in 1152 bytes} [Mon Jul 25 05:37:50 2022] POST /identity/v3/projects => generated 377 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 05:37:50.564568 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2754989-59b5-43ef-86fe-065bc2c19d34 admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:37:50.590881 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ebd352c-2bd6-440a-bdfc-bad19d73a6d8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:37:50.592984 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ebd352c-2bd6-440a-bdfc-bad19d73a6d8 service cinder] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:37:50.603628 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 993/1977] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jul 25 05:37:50 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 167 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 05:37:50.605245 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ebd352c-2bd6-440a-bdfc-bad19d73a6d8 service cinder] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:37:50.615654 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ebd352c-2bd6-440a-bdfc-bad19d73a6d8 service cinder] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:37:50.698262 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2754989-59b5-43ef-86fe-065bc2c19d34 admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:37:50.703596 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76bb8439-e627-46f0-82b8-8c50b1776cf9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:37:50.704551 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76bb8439-e627-46f0-82b8-8c50b1776cf9 admin admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:37:50.705223 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76bb8439-e627-46f0-82b8-8c50b1776cf9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:37:50.705672 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76bb8439-e627-46f0-82b8-8c50b1776cf9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:37:50.706700 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76bb8439-e627-46f0-82b8-8c50b1776cf9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:37:50.708737 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76bb8439-e627-46f0-82b8-8c50b1776cf9 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:37:50.718679 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eaa181a-ddcc-4296-8009-d7ba8853a20f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:37:50.720601 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eaa181a-ddcc-4296-8009-d7ba8853a20f admin admin] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:37:50.721230 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eaa181a-ddcc-4296-8009-d7ba8853a20f admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:37:50.721520 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eaa181a-ddcc-4296-8009-d7ba8853a20f admin admin] PATH_INFO: `/v3/users` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:37:50.722235 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eaa181a-ddcc-4296-8009-d7ba8853a20f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:37:50.724507 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eaa181a-ddcc-4296-8009-d7ba8853a20f admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:37:50.739135 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 985/1978] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jul 25 05:37:50 2022] GET /identity/v3/roles => generated 1420 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 05:37:50.744753 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76bb8439-e627-46f0-82b8-8c50b1776cf9 admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:37:50.759522 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 994/1979] 10.222.0.33 () {60 vars in 1143 bytes} [Mon Jul 25 05:37:50 2022] POST /identity/v3/users => generated 295 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 05:37:50.761005 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eaa181a-ddcc-4296-8009-d7ba8853a20f admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:37:50.767280 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76bb8439-e627-46f0-82b8-8c50b1776cf9 admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:37:50.778589 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4143bbbb-e101-433d-b763-28d0a5c9c924 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:37:50.779272 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4143bbbb-e101-433d-b763-28d0a5c9c924 admin admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:37:50.779621 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4143bbbb-e101-433d-b763-28d0a5c9c924 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:37:50.779901 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4143bbbb-e101-433d-b763-28d0a5c9c924 admin admin] PATH_INFO: `/v3/roles` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:37:50.780637 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4143bbbb-e101-433d-b763-28d0a5c9c924 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:37:50.782391 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4143bbbb-e101-433d-b763-28d0a5c9c924 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:37:50.800584 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 986/1980] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jul 25 05:37:50 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 05:37:50.801971 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4143bbbb-e101-433d-b763-28d0a5c9c924 admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:37:50.814903 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4143bbbb-e101-433d-b763-28d0a5c9c924 admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:37:50.821099 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-565a6307-8425-4e4a-8b8d-274816da2413 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:37:50.822075 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565a6307-8425-4e4a-8b8d-274816da2413 admin admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:37:50.822496 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565a6307-8425-4e4a-8b8d-274816da2413 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:37:50.822951 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565a6307-8425-4e4a-8b8d-274816da2413 admin admin] PATH_INFO: `/v3/roles` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:37:50.823824 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-565a6307-8425-4e4a-8b8d-274816da2413 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:37:50.825790 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-565a6307-8425-4e4a-8b8d-274816da2413 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:37:50.840191 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eaa181a-ddcc-4296-8009-d7ba8853a20f admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:37:50.845759 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-691f6ca4-2a4a-4a99-b2a0-dbe7f82f5689 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db57b478c25643dbaf936f90f6e17d0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92afba266cce4d1bad4134c351e74be3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:37:50.846123 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 987/1981] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jul 25 05:37:50 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 05:37:50.846539 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691f6ca4-2a4a-4a99-b2a0-dbe7f82f5689 service cinder] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:37:50.846837 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691f6ca4-2a4a-4a99-b2a0-dbe7f82f5689 service cinder] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:37:50.847052 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-565a6307-8425-4e4a-8b8d-274816da2413 admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:37:50.847162 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691f6ca4-2a4a-4a99-b2a0-dbe7f82f5689 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:37:50.849835 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-691f6ca4-2a4a-4a99-b2a0-dbe7f82f5689 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:37:50.859479 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-565a6307-8425-4e4a-8b8d-274816da2413 admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:37:50.865303 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86bd8de8-d13c-4502-9a93-d93671b8940b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:37:50.866446 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86bd8de8-d13c-4502-9a93-d93671b8940b admin admin] REQUEST_METHOD: `PUT` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:37:50.867000 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86bd8de8-d13c-4502-9a93-d93671b8940b admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:37:50.867434 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86bd8de8-d13c-4502-9a93-d93671b8940b admin admin] PATH_INFO: `/v3/projects/c7e1aaae19034eed9ff6e4e7b371fa92/users/c76c7c84d7e2487296236a3652f5cf54/roles/5074e74e35f54d4b9bb415c34999a82a` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:37:50.892386 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86bd8de8-d13c-4502-9a93-d93671b8940b admin admin] RBAC: Authorizing `identity:create_grant(project_id=c7e1aaae19034eed9ff6e4e7b371fa92, user_id=c76c7c84d7e2487296236a3652f5cf54, role_id=5074e74e35f54d4b9bb415c34999a82a)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:37:50.895459 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86bd8de8-d13c-4502-9a93-d93671b8940b admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:37:50.916407 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:37:50.932862 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 988/1982] 10.222.0.33 () {60 vars in 1482 bytes} [Mon Jul 25 05:37:50 2022] PUT /identity/v3/projects/c7e1aaae19034eed9ff6e4e7b371fa92/users/c76c7c84d7e2487296236a3652f5cf54/roles/5074e74e35f54d4b9bb415c34999a82a => generated 0 bytes in 78 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 05:37:50.934692 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86bd8de8-d13c-4502-9a93-d93671b8940b admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:37:50.973391 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-691f6ca4-2a4a-4a99-b2a0-dbe7f82f5689 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:37:50.975273 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-691f6ca4-2a4a-4a99-b2a0-dbe7f82f5689 service cinder] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:37:50.986416 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 995/1983] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jul 25 05:37:50 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 226 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 05:37:50.987713 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54edd02-566e-4ff4-a781-23baab1d5e8e None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:37:50.988008 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54edd02-566e-4ff4-a781-23baab1d5e8e None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:37:50.988411 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54edd02-566e-4ff4-a781-23baab1d5e8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:37:51.018340 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:37:51.022378 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-b54edd02-566e-4ff4-a781-23baab1d5e8e None None] MFA Rules not processed for user `c76c7c84d7e2487296236a3652f5cf54`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:37:51.024352 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-b54edd02-566e-4ff4-a781-23baab1d5e8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:37:51.047312 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86bd8de8-d13c-4502-9a93-d93671b8940b admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:37:51.053176 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab974329-adbb-4572-a344-b7491ef9f56e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:37:51.054321 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab974329-adbb-4572-a344-b7491ef9f56e admin admin] REQUEST_METHOD: `PUT` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:37:51.054838 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab974329-adbb-4572-a344-b7491ef9f56e admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:37:51.055249 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab974329-adbb-4572-a344-b7491ef9f56e admin admin] PATH_INFO: `/v3/projects/4852ad6bb4a44edcbc2f797588d5aa84/users/5c7f3ad50365464e8eb77588525ee531/roles/39e5c666285a42fe8fc2c3afca6260d1` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:37:51.075022 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab974329-adbb-4572-a344-b7491ef9f56e admin admin] RBAC: Authorizing `identity:create_grant(project_id=4852ad6bb4a44edcbc2f797588d5aa84, user_id=5c7f3ad50365464e8eb77588525ee531, role_id=39e5c666285a42fe8fc2c3afca6260d1)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:37:51.078305 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab974329-adbb-4572-a344-b7491ef9f56e admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:37:51.106469 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 989/1984] 10.222.0.33 () {60 vars in 1482 bytes} [Mon Jul 25 05:37:50 2022] PUT /identity/v3/projects/4852ad6bb4a44edcbc2f797588d5aa84/users/5c7f3ad50365464e8eb77588525ee531/roles/39e5c666285a42fe8fc2c3afca6260d1 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 05:37:51.110281 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab974329-adbb-4572-a344-b7491ef9f56e admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:37:51.147758 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 996/1985] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 25 05:37:50 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:37:51.152590 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b54edd02-566e-4ff4-a781-23baab1d5e8e None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:37:51.166337 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b54edd02-566e-4ff4-a781-23baab1d5e8e None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:37:51.224833 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab974329-adbb-4572-a344-b7491ef9f56e admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:37:51.240646 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a3806b-7592-4a17-9d7f-bfffea53051b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:37:51.241456 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a3806b-7592-4a17-9d7f-bfffea53051b admin admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:37:51.241735 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a3806b-7592-4a17-9d7f-bfffea53051b admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:37:51.242017 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a3806b-7592-4a17-9d7f-bfffea53051b admin admin] PATH_INFO: `/v3/roles` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:37:51.243986 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6a3806b-7592-4a17-9d7f-bfffea53051b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:37:51.245866 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6a3806b-7592-4a17-9d7f-bfffea53051b admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:37:51.251538 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f6970a-757d-4ed1-af7d-7720358a0c9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:37:51.252976 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f6970a-757d-4ed1-af7d-7720358a0c9e admin admin] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:37:51.253699 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f6970a-757d-4ed1-af7d-7720358a0c9e admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:37:51.254236 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f6970a-757d-4ed1-af7d-7720358a0c9e admin admin] PATH_INFO: `/v3/projects` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:37:51.255035 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8f6970a-757d-4ed1-af7d-7720358a0c9e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:37:51.257119 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8f6970a-757d-4ed1-af7d-7720358a0c9e admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:37:51.275333 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 990/1986] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jul 25 05:37:51 2022] GET /identity/v3/roles => generated 1420 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 05:37:51.280277 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a3806b-7592-4a17-9d7f-bfffea53051b admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:37:51.293889 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a3806b-7592-4a17-9d7f-bfffea53051b admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:37:51.301471 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-401228d6-1167-4552-a149-f88bdd9a02dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:37:51.302632 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401228d6-1167-4552-a149-f88bdd9a02dd admin admin] REQUEST_METHOD: `PUT` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:37:51.302815 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401228d6-1167-4552-a149-f88bdd9a02dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:37:51.309499 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401228d6-1167-4552-a149-f88bdd9a02dd admin admin] PATH_INFO: `/v3/projects/4852ad6bb4a44edcbc2f797588d5aa84/users/5c7f3ad50365464e8eb77588525ee531/roles/39e5c666285a42fe8fc2c3afca6260d1` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:37:51.313059 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 997/1987] 10.222.0.33 () {60 vars in 1152 bytes} [Mon Jul 25 05:37:51 2022] POST /identity/v3/projects => generated 387 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 25 05:37:51.314542 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-401228d6-1167-4552-a149-f88bdd9a02dd admin admin] RBAC: Authorizing `identity:create_grant(project_id=4852ad6bb4a44edcbc2f797588d5aa84, user_id=5c7f3ad50365464e8eb77588525ee531, role_id=39e5c666285a42fe8fc2c3afca6260d1)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:37:51.317519 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-401228d6-1167-4552-a149-f88bdd9a02dd admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:37:51.330284 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f6970a-757d-4ed1-af7d-7720358a0c9e admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:37:51.341856 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 991/1988] 10.222.0.33 () {60 vars in 1482 bytes} [Mon Jul 25 05:37:51 2022] PUT /identity/v3/projects/4852ad6bb4a44edcbc2f797588d5aa84/users/5c7f3ad50365464e8eb77588525ee531/roles/39e5c666285a42fe8fc2c3afca6260d1 => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 25 05:37:51.357227 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d5be6e-f53b-4fd6-b23a-186325918686 None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:37:51.357759 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d5be6e-f53b-4fd6-b23a-186325918686 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:37:51.358262 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d5be6e-f53b-4fd6-b23a-186325918686 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:37:51.408648 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-f0d5be6e-f53b-4fd6-b23a-186325918686 None None] MFA Rules not processed for user `5c7f3ad50365464e8eb77588525ee531`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:37:51.411181 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-f0d5be6e-f53b-4fd6-b23a-186325918686 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:37:51.444310 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:37:51.473122 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f6970a-757d-4ed1-af7d-7720358a0c9e admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:37:51.561809 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 992/1989] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 25 05:37:51 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 05:37:51.572453 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d5be6e-f53b-4fd6-b23a-186325918686 None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:37:51.575214 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c883e11-a14a-4514-af99-ecc3368de7c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:37:51.575946 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c883e11-a14a-4514-af99-ecc3368de7c2 admin admin] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:37:51.576536 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c883e11-a14a-4514-af99-ecc3368de7c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:37:51.576855 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c883e11-a14a-4514-af99-ecc3368de7c2 admin admin] PATH_INFO: `/v3/users` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:37:51.577864 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c883e11-a14a-4514-af99-ecc3368de7c2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:37:51.580265 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c883e11-a14a-4514-af99-ecc3368de7c2 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:37:51.620854 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 998/1990] 10.222.0.33 () {60 vars in 1143 bytes} [Mon Jul 25 05:37:51 2022] POST /identity/v3/users => generated 300 bytes in 301 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 25 05:37:51.621939 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c883e11-a14a-4514-af99-ecc3368de7c2 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:37:51.641240 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c883e11-a14a-4514-af99-ecc3368de7c2 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:37:51.647857 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30458338-61bb-4d5a-9494-d1bd5417c1f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:37:51.648539 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:37:51.648670 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30458338-61bb-4d5a-9494-d1bd5417c1f8 admin admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:37:51.649236 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30458338-61bb-4d5a-9494-d1bd5417c1f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:37:51.649236 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30458338-61bb-4d5a-9494-d1bd5417c1f8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:37:51.658282 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30458338-61bb-4d5a-9494-d1bd5417c1f8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:37:51.667857 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30458338-61bb-4d5a-9494-d1bd5417c1f8 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:37:51.687922 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 999/1991] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jul 25 05:37:51 2022] GET /identity/v3/roles => generated 1420 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 25 05:37:51.689664 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d5be6e-f53b-4fd6-b23a-186325918686 None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:37:51.692627 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30458338-61bb-4d5a-9494-d1bd5417c1f8 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:37:51.696324 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fffde73b-1cd3-4478-81be-599f7c339a19 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db57b478c25643dbaf936f90f6e17d0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92afba266cce4d1bad4134c351e74be3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:37:51.697619 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fffde73b-1cd3-4478-81be-599f7c339a19 service cinder] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:37:51.697619 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fffde73b-1cd3-4478-81be-599f7c339a19 service cinder] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:37:51.698537 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fffde73b-1cd3-4478-81be-599f7c339a19 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:37:51.712076 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30458338-61bb-4d5a-9494-d1bd5417c1f8 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:37:51.719585 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c21080eb-3744-4038-9ee4-4d0da321ab47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:37:51.721109 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21080eb-3744-4038-9ee4-4d0da321ab47 admin admin] REQUEST_METHOD: `PUT` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:37:51.721699 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21080eb-3744-4038-9ee4-4d0da321ab47 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:37:51.722160 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21080eb-3744-4038-9ee4-4d0da321ab47 admin admin] PATH_INFO: `/v3/projects/3f397ce61ef540909aeda972c80a34b7/users/6cdb3b365a0f44739462f84d5ab54979/roles/39e5c666285a42fe8fc2c3afca6260d1` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:37:51.750186 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c21080eb-3744-4038-9ee4-4d0da321ab47 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3f397ce61ef540909aeda972c80a34b7, user_id=6cdb3b365a0f44739462f84d5ab54979, role_id=39e5c666285a42fe8fc2c3afca6260d1)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:37:51.753630 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c21080eb-3744-4038-9ee4-4d0da321ab47 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:37:51.770927 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1000/1992] 10.222.0.33 () {60 vars in 1482 bytes} [Mon Jul 25 05:37:51 2022] PUT /identity/v3/projects/3f397ce61ef540909aeda972c80a34b7/users/6cdb3b365a0f44739462f84d5ab54979/roles/39e5c666285a42fe8fc2c3afca6260d1 => generated 0 bytes in 79 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 05:37:51.776512 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c21080eb-3744-4038-9ee4-4d0da321ab47 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:37:51.800803 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fffde73b-1cd3-4478-81be-599f7c339a19 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:37:51.803212 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fffde73b-1cd3-4478-81be-599f7c339a19 service cinder] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:37:51.819107 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 993/1993] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jul 25 05:37:51 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 247 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 25 05:37:51.889972 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c21080eb-3744-4038-9ee4-4d0da321ab47 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:37:51.894952 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cdfa3d5-1fd7-4043-b478-cd380aaa148f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:37:51.895636 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cdfa3d5-1fd7-4043-b478-cd380aaa148f admin admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:37:51.895922 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cdfa3d5-1fd7-4043-b478-cd380aaa148f admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:37:51.896202 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cdfa3d5-1fd7-4043-b478-cd380aaa148f admin admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:37:51.896974 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cdfa3d5-1fd7-4043-b478-cd380aaa148f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:37:51.898897 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cdfa3d5-1fd7-4043-b478-cd380aaa148f admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:37:51.910711 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1001/1994] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jul 25 05:37:51 2022] GET /identity/v3/roles => generated 1420 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 05:37:51.915036 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fffde73b-1cd3-4478-81be-599f7c339a19 service cinder] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:37:51.930358 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fffde73b-1cd3-4478-81be-599f7c339a19 service cinder] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:37:51.938174 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5353272-872b-4c61-bc36-045d72dd972a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:37:51.939989 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5353272-872b-4c61-bc36-045d72dd972a admin admin] REQUEST_METHOD: `PUT` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:37:51.940535 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5353272-872b-4c61-bc36-045d72dd972a admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:37:51.940981 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5353272-872b-4c61-bc36-045d72dd972a admin admin] PATH_INFO: `/v3/projects/3f397ce61ef540909aeda972c80a34b7/users/6cdb3b365a0f44739462f84d5ab54979/roles/39e5c666285a42fe8fc2c3afca6260d1` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:37:51.945794 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5353272-872b-4c61-bc36-045d72dd972a admin admin] RBAC: Authorizing `identity:create_grant(project_id=3f397ce61ef540909aeda972c80a34b7, user_id=6cdb3b365a0f44739462f84d5ab54979, role_id=39e5c666285a42fe8fc2c3afca6260d1)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:37:51.948922 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5353272-872b-4c61-bc36-045d72dd972a admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:37:51.962818 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 994/1995] 10.222.0.33 () {60 vars in 1482 bytes} [Mon Jul 25 05:37:51 2022] PUT /identity/v3/projects/3f397ce61ef540909aeda972c80a34b7/users/6cdb3b365a0f44739462f84d5ab54979/roles/39e5c666285a42fe8fc2c3afca6260d1 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 25 05:37:51.973625 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b882e5-6f13-4f1f-b1ca-d262d3d6e490 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:37:51.973937 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b882e5-6f13-4f1f-b1ca-d262d3d6e490 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:37:51.974305 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b882e5-6f13-4f1f-b1ca-d262d3d6e490 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:37:52.016226 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-41b882e5-6f13-4f1f-b1ca-d262d3d6e490 None None] MFA Rules not processed for user `6cdb3b365a0f44739462f84d5ab54979`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:37:52.018440 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-41b882e5-6f13-4f1f-b1ca-d262d3d6e490 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:37:52.066303 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:37:52.121151 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1002/1996] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 25 05:37:51 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:37:52.175532 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5353272-872b-4c61-bc36-045d72dd972a admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:37:52.246091 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5353272-872b-4c61-bc36-045d72dd972a admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:37:52.250266 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f6754a-91a2-4ae7-9739-15232806b9a5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db57b478c25643dbaf936f90f6e17d0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92afba266cce4d1bad4134c351e74be3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:37:52.251009 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f6754a-91a2-4ae7-9739-15232806b9a5 service cinder] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:37:52.251304 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f6754a-91a2-4ae7-9739-15232806b9a5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:37:52.251775 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f6754a-91a2-4ae7-9739-15232806b9a5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:37:52.271027 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4f6754a-91a2-4ae7-9739-15232806b9a5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:37:52.272462 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4f6754a-91a2-4ae7-9739-15232806b9a5 service cinder] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:37:52.289490 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 995/1997] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jul 25 05:37:52 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 05:37:52.523095 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b882e5-6f13-4f1f-b1ca-d262d3d6e490 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:37:52.524671 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-41b882e5-6f13-4f1f-b1ca-d262d3d6e490 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:37:52.632007 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b882e5-6f13-4f1f-b1ca-d262d3d6e490 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:37:52.641635 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb174c1-6b1d-4de7-b31a-dc13bfff36c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:37:52.644438 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb174c1-6b1d-4de7-b31a-dc13bfff36c4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:37:52.645141 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb174c1-6b1d-4de7-b31a-dc13bfff36c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:37:52.645429 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb174c1-6b1d-4de7-b31a-dc13bfff36c4 admin admin] PATH_INFO: `/v3/users/6ff9830c0fcd469386cf9c014ac1798a` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:37:52.647351 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddb174c1-6b1d-4de7-b31a-dc13bfff36c4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6ff9830c0fcd469386cf9c014ac1798a)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:37:52.649699 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddb174c1-6b1d-4de7-b31a-dc13bfff36c4 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:37:52.722634 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-ddb174c1-6b1d-4de7-b31a-dc13bfff36c4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6ff9830c0fcd469386cf9c014ac1798a'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:37:52.723380 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-ddb174c1-6b1d-4de7-b31a-dc13bfff36c4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6ff9830c0fcd469386cf9c014ac1798a'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:37:52.737005 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-ddb174c1-6b1d-4de7-b31a-dc13bfff36c4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6ff9830c0fcd469386cf9c014ac1798a'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:37:52.787579 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-ddb174c1-6b1d-4de7-b31a-dc13bfff36c4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6ff9830c0fcd469386cf9c014ac1798a'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:37:52.788576 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-ddb174c1-6b1d-4de7-b31a-dc13bfff36c4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6ff9830c0fcd469386cf9c014ac1798a'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:37:52.829233 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1003/1998] 10.222.0.33 () {58 vars in 1223 bytes} [Mon Jul 25 05:37:52 2022] DELETE /identity/v3/users/6ff9830c0fcd469386cf9c014ac1798a => generated 0 bytes in 304 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 05:37:52.833478 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f6f473-ff38-479e-befc-5ab6c1f6ecef None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:37:52.833942 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f6f473-ff38-479e-befc-5ab6c1f6ecef None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:37:52.834804 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f6f473-ff38-479e-befc-5ab6c1f6ecef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:37:52.868628 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-37f6f473-ff38-479e-befc-5ab6c1f6ecef None None] MFA Rules not processed for user `1eae540242d94c10b0ad33960a855f08`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:37:52.870356 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-37f6f473-ff38-479e-befc-5ab6c1f6ecef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:37:53.002868 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 996/1999] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 25 05:37:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:37:53.012317 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb174c1-6b1d-4de7-b31a-dc13bfff36c4 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:37:53.013775 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-ddb174c1-6b1d-4de7-b31a-dc13bfff36c4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:37:53.037103 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f6f473-ff38-479e-befc-5ab6c1f6ecef None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:37:53.038759 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-37f6f473-ff38-479e-befc-5ab6c1f6ecef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:37:53.076118 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:37:53.101824 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb174c1-6b1d-4de7-b31a-dc13bfff36c4 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:37:53.105887 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae4d3161-d1ea-48e1-b471-a00e67f64f9d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b2931200f99e43d79a50a9f6e02738e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92afba266cce4d1bad4134c351e74be3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:37:53.106673 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4d3161-d1ea-48e1-b471-a00e67f64f9d service neutron] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:37:53.106956 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4d3161-d1ea-48e1-b471-a00e67f64f9d service neutron] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:37:53.107244 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4d3161-d1ea-48e1-b471-a00e67f64f9d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:37:53.114871 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:37:53.117518 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f6f473-ff38-479e-befc-5ab6c1f6ecef None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:37:53.119645 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae4d3161-d1ea-48e1-b471-a00e67f64f9d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:37:53.123376 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae4d3161-d1ea-48e1-b471-a00e67f64f9d service neutron] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:37:53.124545 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f10f6b-d899-42c9-851f-cdace12ab875 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db57b478c25643dbaf936f90f6e17d0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92afba266cce4d1bad4134c351e74be3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:37:53.125411 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f10f6b-d899-42c9-851f-cdace12ab875 service cinder] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:37:53.125892 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f10f6b-d899-42c9-851f-cdace12ab875 service cinder] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:37:53.126217 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f10f6b-d899-42c9-851f-cdace12ab875 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:37:53.128579 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-b8f10f6b-d899-42c9-851f-cdace12ab875 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:37:53.143526 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1004/2000] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jul 25 05:37:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 05:37:53.205387 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f10f6b-d899-42c9-851f-cdace12ab875 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:37:53.206909 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f10f6b-d899-42c9-851f-cdace12ab875 service cinder] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:37:53.228882 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 997/2001] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jul 25 05:37:53 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 192 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 25 05:37:53.533236 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae4d3161-d1ea-48e1-b471-a00e67f64f9d service neutron] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:37:53.544235 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-ae4d3161-d1ea-48e1-b471-a00e67f64f9d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:37:53.574842 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae4d3161-d1ea-48e1-b471-a00e67f64f9d service neutron] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:37:53.607636 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a7afc5-a5f8-4043-b9d3-c1ad69f768f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:37:53.609381 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a7afc5-a5f8-4043-b9d3-c1ad69f768f7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:37:53.609766 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a7afc5-a5f8-4043-b9d3-c1ad69f768f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:37:53.610289 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a7afc5-a5f8-4043-b9d3-c1ad69f768f7 admin admin] PATH_INFO: `/v3/projects/54430781b5024e5e92b8399f9bc37e86` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:37:53.614696 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98a7afc5-a5f8-4043-b9d3-c1ad69f768f7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=54430781b5024e5e92b8399f9bc37e86)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:37:53.621265 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98a7afc5-a5f8-4043-b9d3-c1ad69f768f7 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:37:53.703328 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-98a7afc5-a5f8-4043-b9d3-c1ad69f768f7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '54430781b5024e5e92b8399f9bc37e86', 'request_id': 'req-98a7afc5-a5f8-4043-b9d3-c1ad69f768f7'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:37:53.711836 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-98a7afc5-a5f8-4043-b9d3-c1ad69f768f7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '54430781b5024e5e92b8399f9bc37e86', 'request_id': 'req-98a7afc5-a5f8-4043-b9d3-c1ad69f768f7'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:37:53.720868 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-98a7afc5-a5f8-4043-b9d3-c1ad69f768f7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '54430781b5024e5e92b8399f9bc37e86', 'request_id': 'req-98a7afc5-a5f8-4043-b9d3-c1ad69f768f7'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:37:53.738519 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-98a7afc5-a5f8-4043-b9d3-c1ad69f768f7 admin admin] The token cache is being invalidate because project 54430781b5024e5e92b8399f9bc37e86 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80249) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 05:37:53.738924 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-98a7afc5-a5f8-4043-b9d3-c1ad69f768f7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:37:53.739897 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-98a7afc5-a5f8-4043-b9d3-c1ad69f768f7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:37:53.741923 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1005/2002] 10.222.0.33 () {58 vars in 1232 bytes} [Mon Jul 25 05:37:53 2022] DELETE /identity/v3/projects/54430781b5024e5e92b8399f9bc37e86 => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 05:37:53.746857 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f10f6b-d899-42c9-851f-cdace12ab875 service cinder] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:37:53.759311 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f10f6b-d899-42c9-851f-cdace12ab875 service cinder] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:37:53.859268 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ce5eb2-ac05-482e-a864-f0ebe657fafa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:37:53.861680 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ce5eb2-ac05-482e-a864-f0ebe657fafa admin admin] REQUEST_METHOD: `DELETE` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:37:53.862021 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ce5eb2-ac05-482e-a864-f0ebe657fafa admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:37:53.862308 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ce5eb2-ac05-482e-a864-f0ebe657fafa admin admin] PATH_INFO: `/v3/users/79e4d967bd3f434cb4e0e54fb0bbdbe5` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:37:53.864059 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49ce5eb2-ac05-482e-a864-f0ebe657fafa admin admin] RBAC: Authorizing `identity:delete_user(user_id=79e4d967bd3f434cb4e0e54fb0bbdbe5)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:37:53.866417 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49ce5eb2-ac05-482e-a864-f0ebe657fafa admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:37:53.946229 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-49ce5eb2-ac05-482e-a864-f0ebe657fafa admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '79e4d967bd3f434cb4e0e54fb0bbdbe5'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:37:53.959575 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-49ce5eb2-ac05-482e-a864-f0ebe657fafa admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '79e4d967bd3f434cb4e0e54fb0bbdbe5'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:37:53.960824 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-49ce5eb2-ac05-482e-a864-f0ebe657fafa admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '79e4d967bd3f434cb4e0e54fb0bbdbe5'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:37:53.978684 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-49ce5eb2-ac05-482e-a864-f0ebe657fafa admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '79e4d967bd3f434cb4e0e54fb0bbdbe5'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:37:53.980098 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-49ce5eb2-ac05-482e-a864-f0ebe657fafa admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '79e4d967bd3f434cb4e0e54fb0bbdbe5'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:37:53.989826 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:37:53.998759 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 998/2003] 10.222.0.33 () {58 vars in 1223 bytes} [Mon Jul 25 05:37:53 2022] DELETE /identity/v3/users/79e4d967bd3f434cb4e0e54fb0bbdbe5 => generated 0 bytes in 252 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 05:37:54.291102 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a7afc5-a5f8-4043-b9d3-c1ad69f768f7 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:37:54.292707 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-98a7afc5-a5f8-4043-b9d3-c1ad69f768f7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:37:54.307665 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a7afc5-a5f8-4043-b9d3-c1ad69f768f7 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:37:54.373361 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aea06e4-2793-4443-900d-247edc5256ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:37:54.374322 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aea06e4-2793-4443-900d-247edc5256ca admin admin] REQUEST_METHOD: `DELETE` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:37:54.374689 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aea06e4-2793-4443-900d-247edc5256ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:37:54.374974 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aea06e4-2793-4443-900d-247edc5256ca admin admin] PATH_INFO: `/v3/projects/f504125d6ec1415aa6e75f0a53480ed7` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:37:54.376302 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aea06e4-2793-4443-900d-247edc5256ca admin admin] RBAC: Authorizing `identity:delete_project(project_id=f504125d6ec1415aa6e75f0a53480ed7)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:37:54.378263 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aea06e4-2793-4443-900d-247edc5256ca admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:37:54.424828 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-4aea06e4-2793-4443-900d-247edc5256ca admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f504125d6ec1415aa6e75f0a53480ed7', 'request_id': 'req-4aea06e4-2793-4443-900d-247edc5256ca'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:37:54.430376 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-4aea06e4-2793-4443-900d-247edc5256ca admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f504125d6ec1415aa6e75f0a53480ed7', 'request_id': 'req-4aea06e4-2793-4443-900d-247edc5256ca'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:37:54.437074 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-4aea06e4-2793-4443-900d-247edc5256ca admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f504125d6ec1415aa6e75f0a53480ed7', 'request_id': 'req-4aea06e4-2793-4443-900d-247edc5256ca'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:37:54.450952 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-4aea06e4-2793-4443-900d-247edc5256ca admin admin] The token cache is being invalidate because project f504125d6ec1415aa6e75f0a53480ed7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80249) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 05:37:54.451241 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-4aea06e4-2793-4443-900d-247edc5256ca admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:37:54.451866 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-4aea06e4-2793-4443-900d-247edc5256ca admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:37:54.453537 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1006/2004] 10.222.0.33 () {58 vars in 1232 bytes} [Mon Jul 25 05:37:54 2022] DELETE /identity/v3/projects/f504125d6ec1415aa6e75f0a53480ed7 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 05:37:54.458749 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d54c3a0-f061-4c1f-a5b3-d596a9880bbd None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:37:54.459210 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d54c3a0-f061-4c1f-a5b3-d596a9880bbd None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:37:54.459299 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d54c3a0-f061-4c1f-a5b3-d596a9880bbd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:37:54.477462 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-7d54c3a0-f061-4c1f-a5b3-d596a9880bbd None None] MFA Rules not processed for user `1eae540242d94c10b0ad33960a855f08`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:37:54.478799 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-7d54c3a0-f061-4c1f-a5b3-d596a9880bbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:37:54.537791 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 999/2005] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 25 05:37:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:37:54.547290 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c25a131-7b01-4fb8-8c7e-c1613a8cacb4 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:37:54.547785 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c25a131-7b01-4fb8-8c7e-c1613a8cacb4 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:37:54.547980 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c25a131-7b01-4fb8-8c7e-c1613a8cacb4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:37:54.561482 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:37:54.567172 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-2c25a131-7b01-4fb8-8c7e-c1613a8cacb4 None None] MFA Rules not processed for user `1eae540242d94c10b0ad33960a855f08`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:37:54.569313 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-2c25a131-7b01-4fb8-8c7e-c1613a8cacb4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:37:54.577899 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1007/2006] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 25 05:37:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:37:54.582531 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d54c3a0-f061-4c1f-a5b3-d596a9880bbd None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:37:54.592994 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d54c3a0-f061-4c1f-a5b3-d596a9880bbd None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:37:54.597669 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc22fd2f-b988-487b-9e15-bd48d81ba15c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:37:54.598355 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc22fd2f-b988-487b-9e15-bd48d81ba15c admin admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:37:54.598640 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc22fd2f-b988-487b-9e15-bd48d81ba15c admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:37:54.598938 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc22fd2f-b988-487b-9e15-bd48d81ba15c admin admin] PATH_INFO: `/v3/domains` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:37:54.599700 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc22fd2f-b988-487b-9e15-bd48d81ba15c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:37:54.600011 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc22fd2f-b988-487b-9e15-bd48d81ba15c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:37:54.601640 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc22fd2f-b988-487b-9e15-bd48d81ba15c admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:37:54.614642 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1000/2007] 10.222.0.33 () {58 vars in 1152 bytes} [Mon Jul 25 05:37:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:37:54.618779 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c25a131-7b01-4fb8-8c7e-c1613a8cacb4 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:37:54.628266 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c25a131-7b01-4fb8-8c7e-c1613a8cacb4 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:37:54.634118 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3252139d-fa99-4013-9ea8-5e3805ef3517 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:37:54.634856 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3252139d-fa99-4013-9ea8-5e3805ef3517 admin admin] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:37:54.635144 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3252139d-fa99-4013-9ea8-5e3805ef3517 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:37:54.635562 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3252139d-fa99-4013-9ea8-5e3805ef3517 admin admin] PATH_INFO: `/v3/projects` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:37:54.636287 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3252139d-fa99-4013-9ea8-5e3805ef3517 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:37:54.638233 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3252139d-fa99-4013-9ea8-5e3805ef3517 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:37:54.665192 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1008/2008] 10.222.0.33 () {60 vars in 1152 bytes} [Mon Jul 25 05:37:54 2022] POST /identity/v3/projects => generated 407 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 05:37:54.669962 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc22fd2f-b988-487b-9e15-bd48d81ba15c admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:37:54.680445 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc22fd2f-b988-487b-9e15-bd48d81ba15c admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:37:54.736375 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c958801e-0e97-4e46-a9a9-701469bb2e93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:37:54.737076 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c958801e-0e97-4e46-a9a9-701469bb2e93 admin admin] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:37:54.737371 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c958801e-0e97-4e46-a9a9-701469bb2e93 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:37:54.737651 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c958801e-0e97-4e46-a9a9-701469bb2e93 admin admin] PATH_INFO: `/v3/users` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:37:54.738357 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c958801e-0e97-4e46-a9a9-701469bb2e93 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:37:54.740267 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c958801e-0e97-4e46-a9a9-701469bb2e93 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:37:54.760985 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1001/2009] 10.222.0.33 () {60 vars in 1143 bytes} [Mon Jul 25 05:37:54 2022] POST /identity/v3/users => generated 304 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 25 05:37:54.765337 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3252139d-fa99-4013-9ea8-5e3805ef3517 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:37:54.775417 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3252139d-fa99-4013-9ea8-5e3805ef3517 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:37:54.780221 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cbf6b9b-9105-4ac2-bbde-0239cc0fa1df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:37:54.780910 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cbf6b9b-9105-4ac2-bbde-0239cc0fa1df admin admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:37:54.781196 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cbf6b9b-9105-4ac2-bbde-0239cc0fa1df admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:37:54.781468 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cbf6b9b-9105-4ac2-bbde-0239cc0fa1df admin admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:37:54.782185 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cbf6b9b-9105-4ac2-bbde-0239cc0fa1df admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:37:54.783765 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cbf6b9b-9105-4ac2-bbde-0239cc0fa1df admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:37:54.796042 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1009/2010] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jul 25 05:37:54 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 05:37:54.799993 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c958801e-0e97-4e46-a9a9-701469bb2e93 admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:37:54.811216 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c958801e-0e97-4e46-a9a9-701469bb2e93 admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:37:54.815919 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf4985d-0982-4d19-9522-8d8fa2100e05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:37:54.816584 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf4985d-0982-4d19-9522-8d8fa2100e05 admin admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:37:54.816929 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf4985d-0982-4d19-9522-8d8fa2100e05 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:37:54.817214 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf4985d-0982-4d19-9522-8d8fa2100e05 admin admin] PATH_INFO: `/v3/roles` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:37:54.817948 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cf4985d-0982-4d19-9522-8d8fa2100e05 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:37:54.819541 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cf4985d-0982-4d19-9522-8d8fa2100e05 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:37:54.830665 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1002/2011] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jul 25 05:37:54 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 05:37:54.834813 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cbf6b9b-9105-4ac2-bbde-0239cc0fa1df admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:37:54.845421 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cbf6b9b-9105-4ac2-bbde-0239cc0fa1df admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:37:54.850148 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c7656b-9c5d-40d8-9fe6-f05421560ac5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:37:54.851060 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c7656b-9c5d-40d8-9fe6-f05421560ac5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:37:54.851346 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c7656b-9c5d-40d8-9fe6-f05421560ac5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:37:54.851625 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c7656b-9c5d-40d8-9fe6-f05421560ac5 admin admin] PATH_INFO: `/v3/projects/a3e559e532c04861a3bee63a720e81c7/users/288a5a0ffe8f48d29fc60c6be08f86e8/roles/5074e74e35f54d4b9bb415c34999a82a` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:37:54.867800 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59c7656b-9c5d-40d8-9fe6-f05421560ac5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a3e559e532c04861a3bee63a720e81c7, user_id=288a5a0ffe8f48d29fc60c6be08f86e8, role_id=5074e74e35f54d4b9bb415c34999a82a)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:37:54.870485 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59c7656b-9c5d-40d8-9fe6-f05421560ac5 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:37:54.884098 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1010/2012] 10.222.0.33 () {60 vars in 1482 bytes} [Mon Jul 25 05:37:54 2022] PUT /identity/v3/projects/a3e559e532c04861a3bee63a720e81c7/users/288a5a0ffe8f48d29fc60c6be08f86e8/roles/5074e74e35f54d4b9bb415c34999a82a => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 05:37:54.894758 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a82ca6-33fe-4008-bf9a-424d66b496d0 None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:37:54.895058 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a82ca6-33fe-4008-bf9a-424d66b496d0 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:37:54.895340 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a82ca6-33fe-4008-bf9a-424d66b496d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:37:54.922858 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-12a82ca6-33fe-4008-bf9a-424d66b496d0 None None] MFA Rules not processed for user `288a5a0ffe8f48d29fc60c6be08f86e8`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:37:54.924071 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-12a82ca6-33fe-4008-bf9a-424d66b496d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:37:54.982015 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:37:54.993198 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1003/2013] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 25 05:37:54 2022] POST /identity/v3/auth/tokens => generated 3274 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 05:37:54.997689 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c7656b-9c5d-40d8-9fe6-f05421560ac5 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:37:55.061441 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c7656b-9c5d-40d8-9fe6-f05421560ac5 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:37:55.066212 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9396e5ee-b4df-4936-a1a5-5b64a8259e11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:37:55.066956 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9396e5ee-b4df-4936-a1a5-5b64a8259e11 admin admin] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:37:55.067247 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9396e5ee-b4df-4936-a1a5-5b64a8259e11 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:37:55.067530 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9396e5ee-b4df-4936-a1a5-5b64a8259e11 admin admin] PATH_INFO: `/v3/projects` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:37:55.068343 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9396e5ee-b4df-4936-a1a5-5b64a8259e11 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:37:55.070372 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9396e5ee-b4df-4936-a1a5-5b64a8259e11 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:37:55.100740 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1011/2014] 10.222.0.33 () {60 vars in 1152 bytes} [Mon Jul 25 05:37:54 2022] POST /identity/v3/projects => generated 409 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 05:37:55.102141 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a82ca6-33fe-4008-bf9a-424d66b496d0 None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:37:55.113239 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a82ca6-33fe-4008-bf9a-424d66b496d0 None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:37:55.179018 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749ad754-5dff-4581-a53e-414c964593db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:37:55.180070 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749ad754-5dff-4581-a53e-414c964593db admin admin] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:37:55.180532 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749ad754-5dff-4581-a53e-414c964593db admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:37:55.180949 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749ad754-5dff-4581-a53e-414c964593db admin admin] PATH_INFO: `/v3/users` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:37:55.181787 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-749ad754-5dff-4581-a53e-414c964593db admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:37:55.184101 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-749ad754-5dff-4581-a53e-414c964593db admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:37:55.208139 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1004/2015] 10.222.0.33 () {60 vars in 1143 bytes} [Mon Jul 25 05:37:55 2022] POST /identity/v3/users => generated 311 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 05:37:55.209582 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9396e5ee-b4df-4936-a1a5-5b64a8259e11 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:37:55.222000 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9396e5ee-b4df-4936-a1a5-5b64a8259e11 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:37:55.226670 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c4cf8c7-d007-47cc-9cf2-61358eb7fac6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:37:55.227334 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4cf8c7-d007-47cc-9cf2-61358eb7fac6 admin admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:37:55.227622 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4cf8c7-d007-47cc-9cf2-61358eb7fac6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:37:55.227903 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4cf8c7-d007-47cc-9cf2-61358eb7fac6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:37:55.228634 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c4cf8c7-d007-47cc-9cf2-61358eb7fac6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:37:55.230372 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c4cf8c7-d007-47cc-9cf2-61358eb7fac6 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:37:55.240772 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1012/2016] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jul 25 05:37:55 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 05:37:55.245617 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749ad754-5dff-4581-a53e-414c964593db admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:37:55.256021 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749ad754-5dff-4581-a53e-414c964593db admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:37:55.261316 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e877aa78-b073-46b1-b33b-f7be32b66991 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:37:55.262181 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e877aa78-b073-46b1-b33b-f7be32b66991 admin admin] REQUEST_METHOD: `PUT` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:37:55.262464 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e877aa78-b073-46b1-b33b-f7be32b66991 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:37:55.262737 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e877aa78-b073-46b1-b33b-f7be32b66991 admin admin] PATH_INFO: `/v3/projects/c9fe142be7cc41bd91e806c2455c62d8/users/c47172a0a15e409982894022d75d47ea/roles/39e5c666285a42fe8fc2c3afca6260d1` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:37:55.277298 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e877aa78-b073-46b1-b33b-f7be32b66991 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c9fe142be7cc41bd91e806c2455c62d8, user_id=c47172a0a15e409982894022d75d47ea, role_id=39e5c666285a42fe8fc2c3afca6260d1)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:37:55.279918 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e877aa78-b073-46b1-b33b-f7be32b66991 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:37:55.293660 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1005/2017] 10.222.0.33 () {60 vars in 1482 bytes} [Mon Jul 25 05:37:55 2022] PUT /identity/v3/projects/c9fe142be7cc41bd91e806c2455c62d8/users/c47172a0a15e409982894022d75d47ea/roles/39e5c666285a42fe8fc2c3afca6260d1 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 05:37:55.297515 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c4cf8c7-d007-47cc-9cf2-61358eb7fac6 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:37:55.307928 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c4cf8c7-d007-47cc-9cf2-61358eb7fac6 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:37:55.363351 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e71bf57-2a3b-4d7a-b435-466aaae2ec77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:37:55.363941 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e71bf57-2a3b-4d7a-b435-466aaae2ec77 admin admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:37:55.364227 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e71bf57-2a3b-4d7a-b435-466aaae2ec77 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:37:55.364501 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e71bf57-2a3b-4d7a-b435-466aaae2ec77 admin admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:37:55.365256 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e71bf57-2a3b-4d7a-b435-466aaae2ec77 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:37:55.366855 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e71bf57-2a3b-4d7a-b435-466aaae2ec77 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:37:55.376335 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1013/2018] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jul 25 05:37:55 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 05:37:55.380470 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e877aa78-b073-46b1-b33b-f7be32b66991 admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:37:55.390963 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e877aa78-b073-46b1-b33b-f7be32b66991 admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:37:55.395696 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-801e5386-4cc8-4a7a-a215-ed05d3d73580 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:37:55.396451 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801e5386-4cc8-4a7a-a215-ed05d3d73580 admin admin] REQUEST_METHOD: `PUT` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:37:55.396737 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801e5386-4cc8-4a7a-a215-ed05d3d73580 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:37:55.397051 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801e5386-4cc8-4a7a-a215-ed05d3d73580 admin admin] PATH_INFO: `/v3/projects/c9fe142be7cc41bd91e806c2455c62d8/users/c47172a0a15e409982894022d75d47ea/roles/39e5c666285a42fe8fc2c3afca6260d1` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:37:55.399015 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-801e5386-4cc8-4a7a-a215-ed05d3d73580 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c9fe142be7cc41bd91e806c2455c62d8, user_id=c47172a0a15e409982894022d75d47ea, role_id=39e5c666285a42fe8fc2c3afca6260d1)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:37:55.402291 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-801e5386-4cc8-4a7a-a215-ed05d3d73580 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:37:55.412981 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1006/2019] 10.222.0.33 () {60 vars in 1482 bytes} [Mon Jul 25 05:37:55 2022] PUT /identity/v3/projects/c9fe142be7cc41bd91e806c2455c62d8/users/c47172a0a15e409982894022d75d47ea/roles/39e5c666285a42fe8fc2c3afca6260d1 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 05:37:55.424715 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bcb98de-81b4-41d6-ae2f-e9e107d9dccb None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:37:55.425245 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bcb98de-81b4-41d6-ae2f-e9e107d9dccb None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:37:55.425529 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bcb98de-81b4-41d6-ae2f-e9e107d9dccb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:37:55.449814 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:37:55.452806 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-0bcb98de-81b4-41d6-ae2f-e9e107d9dccb None None] MFA Rules not processed for user `c47172a0a15e409982894022d75d47ea`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:37:55.454857 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-0bcb98de-81b4-41d6-ae2f-e9e107d9dccb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:37:55.525393 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1014/2020] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 25 05:37:55 2022] POST /identity/v3/auth/tokens => generated 3343 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:37:55.536937 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-801e5386-4cc8-4a7a-a215-ed05d3d73580 admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:37:55.538749 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-801e5386-4cc8-4a7a-a215-ed05d3d73580 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:37:55.597315 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-801e5386-4cc8-4a7a-a215-ed05d3d73580 admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:37:55.601126 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92dc2486-a02f-47ed-a39e-41223c26fc06 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db57b478c25643dbaf936f90f6e17d0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92afba266cce4d1bad4134c351e74be3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:37:55.601784 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92dc2486-a02f-47ed-a39e-41223c26fc06 service cinder] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:37:55.602073 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92dc2486-a02f-47ed-a39e-41223c26fc06 service cinder] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:37:55.602355 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92dc2486-a02f-47ed-a39e-41223c26fc06 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:37:55.611599 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92dc2486-a02f-47ed-a39e-41223c26fc06 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:37:55.613095 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92dc2486-a02f-47ed-a39e-41223c26fc06 service cinder] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:37:55.622521 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1007/2021] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jul 25 05:37:55 2022] GET /identity/v3/auth/tokens => generated 3343 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 05:37:55.771672 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bcb98de-81b4-41d6-ae2f-e9e107d9dccb None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:37:55.773218 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-0bcb98de-81b4-41d6-ae2f-e9e107d9dccb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:37:55.785961 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bcb98de-81b4-41d6-ae2f-e9e107d9dccb None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:37:55.789504 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a86184-466c-42d8-af48-caea05ce5342 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db57b478c25643dbaf936f90f6e17d0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92afba266cce4d1bad4134c351e74be3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:37:55.790146 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a86184-466c-42d8-af48-caea05ce5342 service cinder] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:37:55.790435 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a86184-466c-42d8-af48-caea05ce5342 service cinder] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:37:55.790716 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a86184-466c-42d8-af48-caea05ce5342 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:37:55.849145 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7a86184-466c-42d8-af48-caea05ce5342 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:37:55.850586 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7a86184-466c-42d8-af48-caea05ce5342 service cinder] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:37:55.859972 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1015/2022] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jul 25 05:37:55 2022] GET /identity/v3/auth/tokens => generated 3274 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 05:37:57.667278 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92dc2486-a02f-47ed-a39e-41223c26fc06 service cinder] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:37:57.676990 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92dc2486-a02f-47ed-a39e-41223c26fc06 service cinder] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:37:57.680835 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3074905c-1c6c-44dc-8f8b-59ba6e490b7c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db57b478c25643dbaf936f90f6e17d0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92afba266cce4d1bad4134c351e74be3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:37:57.681478 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3074905c-1c6c-44dc-8f8b-59ba6e490b7c service cinder] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:37:57.681767 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3074905c-1c6c-44dc-8f8b-59ba6e490b7c service cinder] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:37:57.682047 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3074905c-1c6c-44dc-8f8b-59ba6e490b7c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:37:57.683803 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-3074905c-1c6c-44dc-8f8b-59ba6e490b7c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:37:57.732589 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:37:57.754337 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3074905c-1c6c-44dc-8f8b-59ba6e490b7c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:37:57.755834 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3074905c-1c6c-44dc-8f8b-59ba6e490b7c service cinder] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:37:57.767236 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1008/2023] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jul 25 05:37:57 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 05:37:58.636680 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a86184-466c-42d8-af48-caea05ce5342 service cinder] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:37:58.639116 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-e7a86184-466c-42d8-af48-caea05ce5342 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:37:58.726952 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a86184-466c-42d8-af48-caea05ce5342 service cinder] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:37:58.732444 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9036e7b4-56dd-4da9-aa6e-356e1b91b981 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:37:58.733361 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9036e7b4-56dd-4da9-aa6e-356e1b91b981 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:37:58.733646 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9036e7b4-56dd-4da9-aa6e-356e1b91b981 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:37:58.733922 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9036e7b4-56dd-4da9-aa6e-356e1b91b981 admin admin] PATH_INFO: `/v3/users/124f328eaeaf498dbd7258524dabd801` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:37:58.735341 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9036e7b4-56dd-4da9-aa6e-356e1b91b981 admin admin] RBAC: Authorizing `identity:delete_user(user_id=124f328eaeaf498dbd7258524dabd801)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:37:58.738290 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9036e7b4-56dd-4da9-aa6e-356e1b91b981 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:37:58.795127 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:37:58.801695 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-9036e7b4-56dd-4da9-aa6e-356e1b91b981 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '124f328eaeaf498dbd7258524dabd801'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:37:58.802611 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-9036e7b4-56dd-4da9-aa6e-356e1b91b981 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '124f328eaeaf498dbd7258524dabd801'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:37:58.810831 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-9036e7b4-56dd-4da9-aa6e-356e1b91b981 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '124f328eaeaf498dbd7258524dabd801'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:37:58.824517 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-9036e7b4-56dd-4da9-aa6e-356e1b91b981 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '124f328eaeaf498dbd7258524dabd801'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:37:58.825921 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-9036e7b4-56dd-4da9-aa6e-356e1b91b981 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '124f328eaeaf498dbd7258524dabd801'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:37:58.839886 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1016/2024] 10.222.0.33 () {58 vars in 1223 bytes} [Mon Jul 25 05:37:58 2022] DELETE /identity/v3/users/124f328eaeaf498dbd7258524dabd801 => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 05:37:58.844677 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aca5cf9-c318-4731-9629-7c5a08e7e3cc None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:37:58.845141 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aca5cf9-c318-4731-9629-7c5a08e7e3cc None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:37:58.845445 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aca5cf9-c318-4731-9629-7c5a08e7e3cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:37:58.863504 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-2aca5cf9-c318-4731-9629-7c5a08e7e3cc None None] MFA Rules not processed for user `1eae540242d94c10b0ad33960a855f08`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:37:58.864805 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-2aca5cf9-c318-4731-9629-7c5a08e7e3cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:37:58.930253 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1009/2025] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 25 05:37:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:37:58.940172 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9036e7b4-56dd-4da9-aa6e-356e1b91b981 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:37:58.941634 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-9036e7b4-56dd-4da9-aa6e-356e1b91b981 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:37:59.009234 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9036e7b4-56dd-4da9-aa6e-356e1b91b981 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:37:59.014298 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f824a3-d3ee-48ce-aab1-5ec84e436e0a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b2931200f99e43d79a50a9f6e02738e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92afba266cce4d1bad4134c351e74be3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:37:59.014969 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f824a3-d3ee-48ce-aab1-5ec84e436e0a service neutron] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:37:59.015267 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f824a3-d3ee-48ce-aab1-5ec84e436e0a service neutron] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:37:59.015548 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f824a3-d3ee-48ce-aab1-5ec84e436e0a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:37:59.026418 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0f824a3-d3ee-48ce-aab1-5ec84e436e0a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:37:59.028672 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0f824a3-d3ee-48ce-aab1-5ec84e436e0a service neutron] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:37:59.041059 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1017/2026] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jul 25 05:37:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 05:37:59.617147 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aca5cf9-c318-4731-9629-7c5a08e7e3cc None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:37:59.618907 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-2aca5cf9-c318-4731-9629-7c5a08e7e3cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:37:59.657955 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aca5cf9-c318-4731-9629-7c5a08e7e3cc None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:37:59.663910 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-528a5848-1ad1-4f8b-921e-e0a17ceb400a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:37:59.665245 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528a5848-1ad1-4f8b-921e-e0a17ceb400a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:37:59.665635 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528a5848-1ad1-4f8b-921e-e0a17ceb400a admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:37:59.666014 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528a5848-1ad1-4f8b-921e-e0a17ceb400a admin admin] PATH_INFO: `/v3/projects/d458ef950ad24306a2be36a2aafd0ad5` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:37:59.668184 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-528a5848-1ad1-4f8b-921e-e0a17ceb400a admin admin] RBAC: Authorizing `identity:delete_project(project_id=d458ef950ad24306a2be36a2aafd0ad5)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:37:59.670360 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-528a5848-1ad1-4f8b-921e-e0a17ceb400a admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:37:59.723080 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-528a5848-1ad1-4f8b-921e-e0a17ceb400a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd458ef950ad24306a2be36a2aafd0ad5', 'request_id': 'req-528a5848-1ad1-4f8b-921e-e0a17ceb400a'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:37:59.728956 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-528a5848-1ad1-4f8b-921e-e0a17ceb400a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd458ef950ad24306a2be36a2aafd0ad5', 'request_id': 'req-528a5848-1ad1-4f8b-921e-e0a17ceb400a'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:37:59.736141 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-528a5848-1ad1-4f8b-921e-e0a17ceb400a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd458ef950ad24306a2be36a2aafd0ad5', 'request_id': 'req-528a5848-1ad1-4f8b-921e-e0a17ceb400a'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:37:59.750458 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-528a5848-1ad1-4f8b-921e-e0a17ceb400a admin admin] The token cache is being invalidate because project d458ef950ad24306a2be36a2aafd0ad5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80248) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 05:37:59.751365 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-528a5848-1ad1-4f8b-921e-e0a17ceb400a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:37:59.752090 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-528a5848-1ad1-4f8b-921e-e0a17ceb400a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:37:59.753793 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1010/2027] 10.222.0.33 () {58 vars in 1232 bytes} [Mon Jul 25 05:37:59 2022] DELETE /identity/v3/projects/d458ef950ad24306a2be36a2aafd0ad5 => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 05:37:59.758110 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f824a3-d3ee-48ce-aab1-5ec84e436e0a service neutron] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:37:59.770152 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f824a3-d3ee-48ce-aab1-5ec84e436e0a service neutron] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:37:59.833717 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bda5a5c-6f38-4ccd-900b-9a382f79a286 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:37:59.834521 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bda5a5c-6f38-4ccd-900b-9a382f79a286 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:37:59.834801 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bda5a5c-6f38-4ccd-900b-9a382f79a286 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:37:59.835077 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bda5a5c-6f38-4ccd-900b-9a382f79a286 admin admin] PATH_INFO: `/v3/users/31a48da0425f41008fd4faba49d9cb4b` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:37:59.836298 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bda5a5c-6f38-4ccd-900b-9a382f79a286 admin admin] RBAC: Authorizing `identity:delete_user(user_id=31a48da0425f41008fd4faba49d9cb4b)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:37:59.838256 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bda5a5c-6f38-4ccd-900b-9a382f79a286 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:37:59.886265 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-7bda5a5c-6f38-4ccd-900b-9a382f79a286 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '31a48da0425f41008fd4faba49d9cb4b'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:37:59.886915 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-7bda5a5c-6f38-4ccd-900b-9a382f79a286 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '31a48da0425f41008fd4faba49d9cb4b'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:37:59.894504 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-7bda5a5c-6f38-4ccd-900b-9a382f79a286 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '31a48da0425f41008fd4faba49d9cb4b'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:37:59.898836 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:DELETE=12,keystone:SELECT=87,keystone:INSERT=1 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:37:59.909855 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-7bda5a5c-6f38-4ccd-900b-9a382f79a286 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '31a48da0425f41008fd4faba49d9cb4b'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:37:59.910659 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-7bda5a5c-6f38-4ccd-900b-9a382f79a286 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '31a48da0425f41008fd4faba49d9cb4b'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:37:59.925931 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1018/2028] 10.222.0.33 () {58 vars in 1223 bytes} [Mon Jul 25 05:37:59 2022] DELETE /identity/v3/users/31a48da0425f41008fd4faba49d9cb4b => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 05:38:00.229616 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-528a5848-1ad1-4f8b-921e-e0a17ceb400a admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:38:00.234307 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-528a5848-1ad1-4f8b-921e-e0a17ceb400a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:38:00.259019 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-528a5848-1ad1-4f8b-921e-e0a17ceb400a admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:38:00.359046 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ca0cd0-f937-4c03-8785-7c922bffa57c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:38:00.359855 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ca0cd0-f937-4c03-8785-7c922bffa57c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:38:00.360142 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ca0cd0-f937-4c03-8785-7c922bffa57c admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:38:00.360422 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ca0cd0-f937-4c03-8785-7c922bffa57c admin admin] PATH_INFO: `/v3/projects/87bf5735962e48ccbd32293dc0d4857e` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:38:00.361686 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0ca0cd0-f937-4c03-8785-7c922bffa57c admin admin] RBAC: Authorizing `identity:delete_project(project_id=87bf5735962e48ccbd32293dc0d4857e)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:38:00.363766 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0ca0cd0-f937-4c03-8785-7c922bffa57c admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:38:00.386648 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:38:00.457487 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-e0ca0cd0-f937-4c03-8785-7c922bffa57c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '87bf5735962e48ccbd32293dc0d4857e', 'request_id': 'req-e0ca0cd0-f937-4c03-8785-7c922bffa57c'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:38:00.463920 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-e0ca0cd0-f937-4c03-8785-7c922bffa57c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '87bf5735962e48ccbd32293dc0d4857e', 'request_id': 'req-e0ca0cd0-f937-4c03-8785-7c922bffa57c'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:38:00.474427 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-e0ca0cd0-f937-4c03-8785-7c922bffa57c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '87bf5735962e48ccbd32293dc0d4857e', 'request_id': 'req-e0ca0cd0-f937-4c03-8785-7c922bffa57c'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:38:00.498129 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-e0ca0cd0-f937-4c03-8785-7c922bffa57c admin admin] The token cache is being invalidate because project 87bf5735962e48ccbd32293dc0d4857e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80248) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 05:38:00.498640 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-e0ca0cd0-f937-4c03-8785-7c922bffa57c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:38:00.499672 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-e0ca0cd0-f937-4c03-8785-7c922bffa57c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:38:00.502484 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1011/2029] 10.222.0.33 () {58 vars in 1232 bytes} [Mon Jul 25 05:38:00 2022] DELETE /identity/v3/projects/87bf5735962e48ccbd32293dc0d4857e => generated 0 bytes in 274 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 05:38:00.506571 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bda5a5c-6f38-4ccd-900b-9a382f79a286 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:38:00.508057 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-7bda5a5c-6f38-4ccd-900b-9a382f79a286 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:38:00.522489 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bda5a5c-6f38-4ccd-900b-9a382f79a286 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:38:00.595599 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db17f6c-04de-448c-a435-dd3553e39648 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:38:00.597762 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db17f6c-04de-448c-a435-dd3553e39648 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:38:00.597762 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db17f6c-04de-448c-a435-dd3553e39648 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:38:00.597762 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db17f6c-04de-448c-a435-dd3553e39648 admin admin] PATH_INFO: `/v3/users/6d398ae141d7415082da8e61051677b1` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:38:00.599660 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9db17f6c-04de-448c-a435-dd3553e39648 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6d398ae141d7415082da8e61051677b1)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:38:00.605179 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9db17f6c-04de-448c-a435-dd3553e39648 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:38:00.661140 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-9db17f6c-04de-448c-a435-dd3553e39648 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6d398ae141d7415082da8e61051677b1'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:38:00.661802 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-9db17f6c-04de-448c-a435-dd3553e39648 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6d398ae141d7415082da8e61051677b1'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:38:00.670869 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-9db17f6c-04de-448c-a435-dd3553e39648 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6d398ae141d7415082da8e61051677b1'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:38:00.694217 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-9db17f6c-04de-448c-a435-dd3553e39648 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6d398ae141d7415082da8e61051677b1'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:38:00.695057 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-9db17f6c-04de-448c-a435-dd3553e39648 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6d398ae141d7415082da8e61051677b1'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:38:00.714658 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1019/2030] 10.222.0.33 () {58 vars in 1223 bytes} [Mon Jul 25 05:38:00 2022] DELETE /identity/v3/users/6d398ae141d7415082da8e61051677b1 => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 25 05:38:01.035248 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ca0cd0-f937-4c03-8785-7c922bffa57c admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:38:01.036806 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-e0ca0cd0-f937-4c03-8785-7c922bffa57c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:38:01.051401 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ca0cd0-f937-4c03-8785-7c922bffa57c admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:38:01.118746 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a15f3d-3e34-4aa3-9d26-e523db9671bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:38:01.119624 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a15f3d-3e34-4aa3-9d26-e523db9671bb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:38:01.119908 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a15f3d-3e34-4aa3-9d26-e523db9671bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:38:01.120185 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a15f3d-3e34-4aa3-9d26-e523db9671bb admin admin] PATH_INFO: `/v3/projects/ad8eaf4b84a747e19aba9dd2737e9541` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:38:01.121531 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58a15f3d-3e34-4aa3-9d26-e523db9671bb admin admin] RBAC: Authorizing `identity:delete_project(project_id=ad8eaf4b84a747e19aba9dd2737e9541)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:38:01.124023 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58a15f3d-3e34-4aa3-9d26-e523db9671bb admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:38:01.170934 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-58a15f3d-3e34-4aa3-9d26-e523db9671bb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ad8eaf4b84a747e19aba9dd2737e9541', 'request_id': 'req-58a15f3d-3e34-4aa3-9d26-e523db9671bb'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:38:01.176286 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-58a15f3d-3e34-4aa3-9d26-e523db9671bb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ad8eaf4b84a747e19aba9dd2737e9541', 'request_id': 'req-58a15f3d-3e34-4aa3-9d26-e523db9671bb'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:38:01.182412 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-58a15f3d-3e34-4aa3-9d26-e523db9671bb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ad8eaf4b84a747e19aba9dd2737e9541', 'request_id': 'req-58a15f3d-3e34-4aa3-9d26-e523db9671bb'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:38:01.196639 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-58a15f3d-3e34-4aa3-9d26-e523db9671bb admin admin] The token cache is being invalidate because project ad8eaf4b84a747e19aba9dd2737e9541 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80248) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 05:38:01.196984 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-58a15f3d-3e34-4aa3-9d26-e523db9671bb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:38:01.197660 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-58a15f3d-3e34-4aa3-9d26-e523db9671bb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:38:01.199233 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1012/2031] 10.222.0.33 () {58 vars in 1232 bytes} [Mon Jul 25 05:38:01 2022] DELETE /identity/v3/projects/ad8eaf4b84a747e19aba9dd2737e9541 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 05:38:01.206258 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58262c01-362e-4ae6-91fe-eda4b7f765cc None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:38:01.206701 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58262c01-362e-4ae6-91fe-eda4b7f765cc None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:38:01.206825 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58262c01-362e-4ae6-91fe-eda4b7f765cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:38:01.229434 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-58262c01-362e-4ae6-91fe-eda4b7f765cc None None] MFA Rules not processed for user `1eae540242d94c10b0ad33960a855f08`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:38:01.230679 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-58262c01-362e-4ae6-91fe-eda4b7f765cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:38:01.285456 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:DELETE=13,keystone:SELECT=85,keystone:INSERT=2 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:38:01.296291 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1020/2032] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 25 05:38:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:38:01.303765 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21aa440c-8d2b-4f04-803b-3b1c0deba37a None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:38:01.304275 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21aa440c-8d2b-4f04-803b-3b1c0deba37a None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:38:01.304407 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21aa440c-8d2b-4f04-803b-3b1c0deba37a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:38:01.323048 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-21aa440c-8d2b-4f04-803b-3b1c0deba37a None None] MFA Rules not processed for user `1eae540242d94c10b0ad33960a855f08`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:38:01.324256 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-21aa440c-8d2b-4f04-803b-3b1c0deba37a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:38:01.334032 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1013/2033] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 25 05:38:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:38:01.338231 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58262c01-362e-4ae6-91fe-eda4b7f765cc None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:38:01.348741 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58262c01-362e-4ae6-91fe-eda4b7f765cc None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:38:01.353311 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc0a3c8-d58c-4384-bfd9-f52384913551 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:38:01.353978 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc0a3c8-d58c-4384-bfd9-f52384913551 admin admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:38:01.354264 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc0a3c8-d58c-4384-bfd9-f52384913551 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:38:01.354539 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc0a3c8-d58c-4384-bfd9-f52384913551 admin admin] PATH_INFO: `/v3/domains` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:38:01.355306 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbc0a3c8-d58c-4384-bfd9-f52384913551 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:38:01.355613 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbc0a3c8-d58c-4384-bfd9-f52384913551 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:38:01.357415 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbc0a3c8-d58c-4384-bfd9-f52384913551 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:38:01.369398 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1021/2034] 10.222.0.33 () {58 vars in 1152 bytes} [Mon Jul 25 05:38:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:38:01.374337 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21aa440c-8d2b-4f04-803b-3b1c0deba37a None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:38:01.383663 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21aa440c-8d2b-4f04-803b-3b1c0deba37a None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:38:01.389255 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c6ebe35-ae0f-45fe-8595-520f19cc38af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:38:01.389924 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6ebe35-ae0f-45fe-8595-520f19cc38af admin admin] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:38:01.390215 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6ebe35-ae0f-45fe-8595-520f19cc38af admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:38:01.390499 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6ebe35-ae0f-45fe-8595-520f19cc38af admin admin] PATH_INFO: `/v3/projects` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:38:01.391202 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c6ebe35-ae0f-45fe-8595-520f19cc38af admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:38:01.393154 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c6ebe35-ae0f-45fe-8595-520f19cc38af admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:38:01.411943 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:38:01.424081 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1014/2035] 10.222.0.33 () {60 vars in 1152 bytes} [Mon Jul 25 05:38:01 2022] POST /identity/v3/projects => generated 365 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 05:38:01.425309 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc0a3c8-d58c-4384-bfd9-f52384913551 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:38:01.436965 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc0a3c8-d58c-4384-bfd9-f52384913551 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:38:01.520815 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b6176a-9244-4e89-8f92-7cf7e52b3bf2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:38:01.521537 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b6176a-9244-4e89-8f92-7cf7e52b3bf2 admin admin] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:38:01.521831 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b6176a-9244-4e89-8f92-7cf7e52b3bf2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:38:01.522113 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b6176a-9244-4e89-8f92-7cf7e52b3bf2 admin admin] PATH_INFO: `/v3/users` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:38:01.522830 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4b6176a-9244-4e89-8f92-7cf7e52b3bf2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:38:01.525017 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4b6176a-9244-4e89-8f92-7cf7e52b3bf2 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:38:01.564754 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1022/2036] 10.222.0.33 () {60 vars in 1143 bytes} [Mon Jul 25 05:38:01 2022] POST /identity/v3/users => generated 283 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 05:38:01.566410 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c6ebe35-ae0f-45fe-8595-520f19cc38af admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:38:01.582314 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c6ebe35-ae0f-45fe-8595-520f19cc38af admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:38:01.596596 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc9117a-456a-4578-bfaa-88949a7ded44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:38:01.598269 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc9117a-456a-4578-bfaa-88949a7ded44 admin admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:38:01.598718 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc9117a-456a-4578-bfaa-88949a7ded44 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:38:01.599223 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc9117a-456a-4578-bfaa-88949a7ded44 admin admin] PATH_INFO: `/v3/roles` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:38:01.600122 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dc9117a-456a-4578-bfaa-88949a7ded44 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:38:01.602181 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dc9117a-456a-4578-bfaa-88949a7ded44 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:38:01.618330 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1015/2037] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jul 25 05:38:01 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 05:38:01.619458 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b6176a-9244-4e89-8f92-7cf7e52b3bf2 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:38:01.636236 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b6176a-9244-4e89-8f92-7cf7e52b3bf2 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:38:01.642797 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e23861-9a86-4419-afc2-4d2cf4853d74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:38:01.643553 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e23861-9a86-4419-afc2-4d2cf4853d74 admin admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:38:01.643798 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e23861-9a86-4419-afc2-4d2cf4853d74 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:38:01.644199 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e23861-9a86-4419-afc2-4d2cf4853d74 admin admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:38:01.644880 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50e23861-9a86-4419-afc2-4d2cf4853d74 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:38:01.646484 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50e23861-9a86-4419-afc2-4d2cf4853d74 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:38:01.660998 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1023/2038] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jul 25 05:38:01 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 25 05:38:01.662628 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc9117a-456a-4578-bfaa-88949a7ded44 admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:38:01.674205 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc9117a-456a-4578-bfaa-88949a7ded44 admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:38:01.678943 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67bd6124-450f-4994-a0a4-d93d53abe6cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:38:01.679993 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67bd6124-450f-4994-a0a4-d93d53abe6cf admin admin] REQUEST_METHOD: `PUT` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:38:01.680279 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67bd6124-450f-4994-a0a4-d93d53abe6cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:38:01.680561 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67bd6124-450f-4994-a0a4-d93d53abe6cf admin admin] PATH_INFO: `/v3/projects/f4e242dbc34a4321ac10a934cad9e5a0/users/c2ebc24a8b2f4db9ae7f7effa87cca5e/roles/5074e74e35f54d4b9bb415c34999a82a` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:38:01.697753 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67bd6124-450f-4994-a0a4-d93d53abe6cf admin admin] RBAC: Authorizing `identity:create_grant(project_id=f4e242dbc34a4321ac10a934cad9e5a0, user_id=c2ebc24a8b2f4db9ae7f7effa87cca5e, role_id=5074e74e35f54d4b9bb415c34999a82a)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:38:01.700562 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67bd6124-450f-4994-a0a4-d93d53abe6cf admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:38:01.714761 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1016/2039] 10.222.0.33 () {60 vars in 1482 bytes} [Mon Jul 25 05:38:01 2022] PUT /identity/v3/projects/f4e242dbc34a4321ac10a934cad9e5a0/users/c2ebc24a8b2f4db9ae7f7effa87cca5e/roles/5074e74e35f54d4b9bb415c34999a82a => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 05:38:01.726140 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d19f0d-fefa-4645-83db-106ebca2b10f None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:38:01.726437 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d19f0d-fefa-4645-83db-106ebca2b10f None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:38:01.726718 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d19f0d-fefa-4645-83db-106ebca2b10f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:38:01.760603 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-c9d19f0d-fefa-4645-83db-106ebca2b10f None None] MFA Rules not processed for user `c2ebc24a8b2f4db9ae7f7effa87cca5e`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:38:01.761856 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-c9d19f0d-fefa-4645-83db-106ebca2b10f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:38:01.850352 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1024/2040] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 25 05:38:01 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:38:01.863084 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67bd6124-450f-4994-a0a4-d93d53abe6cf admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:38:01.864977 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-67bd6124-450f-4994-a0a4-d93d53abe6cf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:38:01.925667 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67bd6124-450f-4994-a0a4-d93d53abe6cf admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:38:01.929211 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c651dd38-7b99-460a-a040-ca70856b8301 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db57b478c25643dbaf936f90f6e17d0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92afba266cce4d1bad4134c351e74be3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:38:01.929863 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c651dd38-7b99-460a-a040-ca70856b8301 service cinder] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:38:01.930156 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c651dd38-7b99-460a-a040-ca70856b8301 service cinder] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:38:01.930440 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c651dd38-7b99-460a-a040-ca70856b8301 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:38:01.939150 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c651dd38-7b99-460a-a040-ca70856b8301 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:38:01.940597 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c651dd38-7b99-460a-a040-ca70856b8301 service cinder] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:38:01.950633 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1017/2041] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jul 25 05:38:01 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 05:38:05.975865 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d19f0d-fefa-4645-83db-106ebca2b10f None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:38:05.988960 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d19f0d-fefa-4645-83db-106ebca2b10f None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:38:05.994489 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b657557d-8757-41e0-a861-4ea6c8a970f2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db57b478c25643dbaf936f90f6e17d0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92afba266cce4d1bad4134c351e74be3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:38:05.995309 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b657557d-8757-41e0-a861-4ea6c8a970f2 service cinder] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:38:05.995596 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b657557d-8757-41e0-a861-4ea6c8a970f2 service cinder] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:38:05.995879 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b657557d-8757-41e0-a861-4ea6c8a970f2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:38:05.997963 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-b657557d-8757-41e0-a861-4ea6c8a970f2 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:38:06.050006 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:38:06.062691 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b657557d-8757-41e0-a861-4ea6c8a970f2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:38:06.064632 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b657557d-8757-41e0-a861-4ea6c8a970f2 service cinder] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:38:06.074634 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1025/2042] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jul 25 05:38:05 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 05:38:11.648113 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c651dd38-7b99-460a-a040-ca70856b8301 service cinder] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:38:11.649861 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-c651dd38-7b99-460a-a040-ca70856b8301 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:38:11.724057 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c651dd38-7b99-460a-a040-ca70856b8301 service cinder] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:38:11.729568 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb994cd-4260-41e7-9117-577f6582a9a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:38:11.730610 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb994cd-4260-41e7-9117-577f6582a9a4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:38:11.730902 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb994cd-4260-41e7-9117-577f6582a9a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:38:11.731252 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb994cd-4260-41e7-9117-577f6582a9a4 admin admin] PATH_INFO: `/v3/users/4d29be9a17894bb4b18f5f7692a1f8aa` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:38:11.732582 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adb994cd-4260-41e7-9117-577f6582a9a4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4d29be9a17894bb4b18f5f7692a1f8aa)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:38:11.734591 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adb994cd-4260-41e7-9117-577f6582a9a4 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:38:11.787576 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-adb994cd-4260-41e7-9117-577f6582a9a4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4d29be9a17894bb4b18f5f7692a1f8aa'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:38:11.804380 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-adb994cd-4260-41e7-9117-577f6582a9a4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4d29be9a17894bb4b18f5f7692a1f8aa'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:38:11.805238 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-adb994cd-4260-41e7-9117-577f6582a9a4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4d29be9a17894bb4b18f5f7692a1f8aa'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:38:11.808506 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=89,keystone:INSERT=2,keystone:DELETE=9 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:38:11.820944 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-adb994cd-4260-41e7-9117-577f6582a9a4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4d29be9a17894bb4b18f5f7692a1f8aa'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:38:11.821757 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-adb994cd-4260-41e7-9117-577f6582a9a4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4d29be9a17894bb4b18f5f7692a1f8aa'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:38:11.837366 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1018/2043] 10.222.0.33 () {58 vars in 1223 bytes} [Mon Jul 25 05:38:11 2022] DELETE /identity/v3/users/4d29be9a17894bb4b18f5f7692a1f8aa => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 05:38:11.842508 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32ad54a-a1e8-4f59-b9c4-c6588fa07302 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:38:11.843029 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32ad54a-a1e8-4f59-b9c4-c6588fa07302 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:38:11.843332 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32ad54a-a1e8-4f59-b9c4-c6588fa07302 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:38:11.860965 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-b32ad54a-a1e8-4f59-b9c4-c6588fa07302 None None] MFA Rules not processed for user `1eae540242d94c10b0ad33960a855f08`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:38:11.862168 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-b32ad54a-a1e8-4f59-b9c4-c6588fa07302 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:38:11.930435 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1026/2044] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 25 05:38:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:38:11.941103 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb994cd-4260-41e7-9117-577f6582a9a4 admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:38:11.942681 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-adb994cd-4260-41e7-9117-577f6582a9a4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:38:12.022365 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb994cd-4260-41e7-9117-577f6582a9a4 admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:38:12.026177 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89934657-f3e6-4df7-a63d-7389476c7b28 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b2931200f99e43d79a50a9f6e02738e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92afba266cce4d1bad4134c351e74be3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:38:12.026922 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89934657-f3e6-4df7-a63d-7389476c7b28 service neutron] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:38:12.027197 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89934657-f3e6-4df7-a63d-7389476c7b28 service neutron] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:38:12.027605 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89934657-f3e6-4df7-a63d-7389476c7b28 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:38:12.037742 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89934657-f3e6-4df7-a63d-7389476c7b28 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:38:12.039192 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89934657-f3e6-4df7-a63d-7389476c7b28 service neutron] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:38:12.051131 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1019/2045] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jul 25 05:38:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 05:38:12.266293 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b32ad54a-a1e8-4f59-b9c4-c6588fa07302 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:38:12.267727 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-b32ad54a-a1e8-4f59-b9c4-c6588fa07302 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:38:12.280556 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b32ad54a-a1e8-4f59-b9c4-c6588fa07302 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:38:12.286070 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1864ab1-eb43-42f0-8870-afb7e9b71f57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:38:12.286997 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1864ab1-eb43-42f0-8870-afb7e9b71f57 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:38:12.287286 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1864ab1-eb43-42f0-8870-afb7e9b71f57 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:38:12.287565 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1864ab1-eb43-42f0-8870-afb7e9b71f57 admin admin] PATH_INFO: `/v3/users/288a5a0ffe8f48d29fc60c6be08f86e8` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:38:12.288829 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1864ab1-eb43-42f0-8870-afb7e9b71f57 admin admin] RBAC: Authorizing `identity:delete_user(user_id=288a5a0ffe8f48d29fc60c6be08f86e8)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:38:12.290783 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1864ab1-eb43-42f0-8870-afb7e9b71f57 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:38:12.359034 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-e1864ab1-eb43-42f0-8870-afb7e9b71f57 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '288a5a0ffe8f48d29fc60c6be08f86e8'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:38:12.360069 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-e1864ab1-eb43-42f0-8870-afb7e9b71f57 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '288a5a0ffe8f48d29fc60c6be08f86e8'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:38:12.369608 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-e1864ab1-eb43-42f0-8870-afb7e9b71f57 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '288a5a0ffe8f48d29fc60c6be08f86e8'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:38:12.378954 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89934657-f3e6-4df7-a63d-7389476c7b28 service neutron] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:38:12.380651 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-89934657-f3e6-4df7-a63d-7389476c7b28 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:38:12.395535 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-e1864ab1-eb43-42f0-8870-afb7e9b71f57 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '288a5a0ffe8f48d29fc60c6be08f86e8'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:38:12.396500 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-e1864ab1-eb43-42f0-8870-afb7e9b71f57 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '288a5a0ffe8f48d29fc60c6be08f86e8'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:38:12.402687 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89934657-f3e6-4df7-a63d-7389476c7b28 service neutron] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:38:12.409402 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3795bdc5-7f68-47ee-8b6b-e078257c0360 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:38:12.410489 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3795bdc5-7f68-47ee-8b6b-e078257c0360 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:38:12.410842 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3795bdc5-7f68-47ee-8b6b-e078257c0360 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:38:12.411194 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3795bdc5-7f68-47ee-8b6b-e078257c0360 admin admin] PATH_INFO: `/v3/projects/c9bf4bcb38f74a488f2074c00874a56f` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:38:12.412519 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3795bdc5-7f68-47ee-8b6b-e078257c0360 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c9bf4bcb38f74a488f2074c00874a56f)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:38:12.414844 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3795bdc5-7f68-47ee-8b6b-e078257c0360 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:38:12.419676 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1027/2046] 10.222.0.33 () {58 vars in 1223 bytes} [Mon Jul 25 05:38:12 2022] DELETE /identity/v3/users/288a5a0ffe8f48d29fc60c6be08f86e8 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 05:38:12.424373 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6cc070-47fa-4f33-aea5-2a7527209ef7 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:38:12.424373 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6cc070-47fa-4f33-aea5-2a7527209ef7 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:38:12.424373 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6cc070-47fa-4f33-aea5-2a7527209ef7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:38:12.455980 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-6e6cc070-47fa-4f33-aea5-2a7527209ef7 None None] MFA Rules not processed for user `1eae540242d94c10b0ad33960a855f08`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:38:12.457323 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-6e6cc070-47fa-4f33-aea5-2a7527209ef7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:38:12.486053 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-3795bdc5-7f68-47ee-8b6b-e078257c0360 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c9bf4bcb38f74a488f2074c00874a56f', 'request_id': 'req-3795bdc5-7f68-47ee-8b6b-e078257c0360'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:38:12.492234 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-3795bdc5-7f68-47ee-8b6b-e078257c0360 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c9bf4bcb38f74a488f2074c00874a56f', 'request_id': 'req-3795bdc5-7f68-47ee-8b6b-e078257c0360'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:38:12.499381 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-3795bdc5-7f68-47ee-8b6b-e078257c0360 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c9bf4bcb38f74a488f2074c00874a56f', 'request_id': 'req-3795bdc5-7f68-47ee-8b6b-e078257c0360'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:38:12.521137 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-3795bdc5-7f68-47ee-8b6b-e078257c0360 admin admin] The token cache is being invalidate because project c9bf4bcb38f74a488f2074c00874a56f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80248) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 05:38:12.521595 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-3795bdc5-7f68-47ee-8b6b-e078257c0360 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:38:12.523747 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-3795bdc5-7f68-47ee-8b6b-e078257c0360 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:38:12.528765 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1020/2047] 10.222.0.33 () {58 vars in 1232 bytes} [Mon Jul 25 05:38:12 2022] DELETE /identity/v3/projects/c9bf4bcb38f74a488f2074c00874a56f => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 05:38:12.532817 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3795bdc5-7f68-47ee-8b6b-e078257c0360 admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:38:12.536745 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-3795bdc5-7f68-47ee-8b6b-e078257c0360 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:38:12.562161 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1028/2048] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 25 05:38:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:38:12.570446 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e6cc070-47fa-4f33-aea5-2a7527209ef7 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:38:12.573699 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-6e6cc070-47fa-4f33-aea5-2a7527209ef7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:38:12.584156 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:38:12.657241 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e6cc070-47fa-4f33-aea5-2a7527209ef7 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:38:12.659807 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3795bdc5-7f68-47ee-8b6b-e078257c0360 admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:38:12.668983 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adbcf74c-5d76-4669-95cd-67075e2965a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:38:12.669988 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adbcf74c-5d76-4669-95cd-67075e2965a7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:38:12.670839 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adbcf74c-5d76-4669-95cd-67075e2965a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:38:12.671379 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adbcf74c-5d76-4669-95cd-67075e2965a7 admin admin] PATH_INFO: `/v3/users/c76c7c84d7e2487296236a3652f5cf54` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:38:12.678453 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adbcf74c-5d76-4669-95cd-67075e2965a7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c76c7c84d7e2487296236a3652f5cf54)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:38:12.680481 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adbcf74c-5d76-4669-95cd-67075e2965a7 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:38:12.702250 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:DELETE=9,keystone:SELECT=90,keystone:INSERT=1 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:38:12.734294 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e074e7-7d69-4974-864d-dbdfbe959333 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b2931200f99e43d79a50a9f6e02738e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92afba266cce4d1bad4134c351e74be3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:38:12.734979 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e074e7-7d69-4974-864d-dbdfbe959333 service neutron] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:38:12.735264 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e074e7-7d69-4974-864d-dbdfbe959333 service neutron] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:38:12.735688 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e074e7-7d69-4974-864d-dbdfbe959333 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:38:12.748193 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82e074e7-7d69-4974-864d-dbdfbe959333 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:38:12.750247 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82e074e7-7d69-4974-864d-dbdfbe959333 service neutron] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:38:12.764477 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1029/2049] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jul 25 05:38:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 195 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 05:38:12.775045 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-adbcf74c-5d76-4669-95cd-67075e2965a7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c76c7c84d7e2487296236a3652f5cf54'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:38:12.790262 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-adbcf74c-5d76-4669-95cd-67075e2965a7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c76c7c84d7e2487296236a3652f5cf54'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:38:12.791093 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-adbcf74c-5d76-4669-95cd-67075e2965a7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c76c7c84d7e2487296236a3652f5cf54'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:38:12.816618 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-adbcf74c-5d76-4669-95cd-67075e2965a7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c76c7c84d7e2487296236a3652f5cf54'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:38:12.817496 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-adbcf74c-5d76-4669-95cd-67075e2965a7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c76c7c84d7e2487296236a3652f5cf54'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:38:12.833198 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1021/2050] 10.222.0.33 () {58 vars in 1223 bytes} [Mon Jul 25 05:38:12 2022] DELETE /identity/v3/users/c76c7c84d7e2487296236a3652f5cf54 => generated 0 bytes in 301 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 05:38:13.107357 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e074e7-7d69-4974-864d-dbdfbe959333 service neutron] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:38:13.108798 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-82e074e7-7d69-4974-864d-dbdfbe959333 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:38:13.122842 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e074e7-7d69-4974-864d-dbdfbe959333 service neutron] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:38:13.177956 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adbcf74c-5d76-4669-95cd-67075e2965a7 admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:38:13.179392 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-adbcf74c-5d76-4669-95cd-67075e2965a7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:38:13.196893 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a34548-1601-4486-baf3-dc85a55297c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:38:13.197964 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a34548-1601-4486-baf3-dc85a55297c4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:38:13.198412 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a34548-1601-4486-baf3-dc85a55297c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:38:13.198832 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a34548-1601-4486-baf3-dc85a55297c4 admin admin] PATH_INFO: `/v3/projects/a3e559e532c04861a3bee63a720e81c7` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:38:13.200244 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3a34548-1601-4486-baf3-dc85a55297c4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a3e559e532c04861a3bee63a720e81c7)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:38:13.202195 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3a34548-1601-4486-baf3-dc85a55297c4 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:38:13.254688 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-c3a34548-1601-4486-baf3-dc85a55297c4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a3e559e532c04861a3bee63a720e81c7', 'request_id': 'req-c3a34548-1601-4486-baf3-dc85a55297c4'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:38:13.255480 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adbcf74c-5d76-4669-95cd-67075e2965a7 admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:38:13.260201 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-c3a34548-1601-4486-baf3-dc85a55297c4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a3e559e532c04861a3bee63a720e81c7', 'request_id': 'req-c3a34548-1601-4486-baf3-dc85a55297c4'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:38:13.267840 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-c3a34548-1601-4486-baf3-dc85a55297c4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a3e559e532c04861a3bee63a720e81c7', 'request_id': 'req-c3a34548-1601-4486-baf3-dc85a55297c4'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:38:13.270495 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=93,keystone:DELETE=6,keystone:INSERT=1 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:38:13.284361 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-c3a34548-1601-4486-baf3-dc85a55297c4 admin admin] The token cache is being invalidate because project a3e559e532c04861a3bee63a720e81c7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80249) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 05:38:13.284656 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-c3a34548-1601-4486-baf3-dc85a55297c4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:38:13.285348 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-c3a34548-1601-4486-baf3-dc85a55297c4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:38:13.286918 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1030/2051] 10.222.0.33 () {58 vars in 1232 bytes} [Mon Jul 25 05:38:13 2022] DELETE /identity/v3/projects/a3e559e532c04861a3bee63a720e81c7 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 05:38:13.290819 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a34548-1601-4486-baf3-dc85a55297c4 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:38:13.292170 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-c3a34548-1601-4486-baf3-dc85a55297c4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:38:13.315532 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6811a5a-e574-412a-a2e8-b3e8eab46bb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:38:13.316401 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6811a5a-e574-412a-a2e8-b3e8eab46bb7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:38:13.316690 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6811a5a-e574-412a-a2e8-b3e8eab46bb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:38:13.316997 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6811a5a-e574-412a-a2e8-b3e8eab46bb7 admin admin] PATH_INFO: `/v3/projects/c7e1aaae19034eed9ff6e4e7b371fa92` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:38:13.318196 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6811a5a-e574-412a-a2e8-b3e8eab46bb7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c7e1aaae19034eed9ff6e4e7b371fa92)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:38:13.320148 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6811a5a-e574-412a-a2e8-b3e8eab46bb7 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:38:13.357349 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a34548-1601-4486-baf3-dc85a55297c4 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:38:13.361198 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:38:13.370901 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-d6811a5a-e574-412a-a2e8-b3e8eab46bb7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c7e1aaae19034eed9ff6e4e7b371fa92', 'request_id': 'req-d6811a5a-e574-412a-a2e8-b3e8eab46bb7'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:38:13.377511 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-d6811a5a-e574-412a-a2e8-b3e8eab46bb7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c7e1aaae19034eed9ff6e4e7b371fa92', 'request_id': 'req-d6811a5a-e574-412a-a2e8-b3e8eab46bb7'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:38:13.383694 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-d6811a5a-e574-412a-a2e8-b3e8eab46bb7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c7e1aaae19034eed9ff6e4e7b371fa92', 'request_id': 'req-d6811a5a-e574-412a-a2e8-b3e8eab46bb7'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:38:13.396742 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-d6811a5a-e574-412a-a2e8-b3e8eab46bb7 admin admin] The token cache is being invalidate because project c7e1aaae19034eed9ff6e4e7b371fa92 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80248) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 05:38:13.397064 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-d6811a5a-e574-412a-a2e8-b3e8eab46bb7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:38:13.397681 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-d6811a5a-e574-412a-a2e8-b3e8eab46bb7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:38:13.399675 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1022/2052] 10.222.0.33 () {58 vars in 1232 bytes} [Mon Jul 25 05:38:13 2022] DELETE /identity/v3/projects/c7e1aaae19034eed9ff6e4e7b371fa92 => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 05:38:13.404679 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6811a5a-e574-412a-a2e8-b3e8eab46bb7 admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:38:13.406057 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-d6811a5a-e574-412a-a2e8-b3e8eab46bb7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:38:13.418403 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce4e516-cc90-41ac-b7b3-b8ab2554dbb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:38:13.419379 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce4e516-cc90-41ac-b7b3-b8ab2554dbb8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:38:13.419667 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce4e516-cc90-41ac-b7b3-b8ab2554dbb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:38:13.419946 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce4e516-cc90-41ac-b7b3-b8ab2554dbb8 admin admin] PATH_INFO: `/v3/users/c47172a0a15e409982894022d75d47ea` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:38:13.421290 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dce4e516-cc90-41ac-b7b3-b8ab2554dbb8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c47172a0a15e409982894022d75d47ea)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:38:13.423219 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dce4e516-cc90-41ac-b7b3-b8ab2554dbb8 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:38:13.470385 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-dce4e516-cc90-41ac-b7b3-b8ab2554dbb8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c47172a0a15e409982894022d75d47ea'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:38:13.471617 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-dce4e516-cc90-41ac-b7b3-b8ab2554dbb8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c47172a0a15e409982894022d75d47ea'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:38:13.474858 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6811a5a-e574-412a-a2e8-b3e8eab46bb7 admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:38:13.477941 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-dce4e516-cc90-41ac-b7b3-b8ab2554dbb8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c47172a0a15e409982894022d75d47ea'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:38:13.479588 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-752448dd-1936-4051-be8f-0b55b2f941df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:38:13.480450 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752448dd-1936-4051-be8f-0b55b2f941df admin admin] REQUEST_METHOD: `DELETE` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:38:13.480736 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752448dd-1936-4051-be8f-0b55b2f941df admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:38:13.481104 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752448dd-1936-4051-be8f-0b55b2f941df admin admin] PATH_INFO: `/v3/users/6cdb3b365a0f44739462f84d5ab54979` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:38:13.482810 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-752448dd-1936-4051-be8f-0b55b2f941df admin admin] RBAC: Authorizing `identity:delete_user(user_id=6cdb3b365a0f44739462f84d5ab54979)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:38:13.484727 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-752448dd-1936-4051-be8f-0b55b2f941df admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:38:13.490232 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-dce4e516-cc90-41ac-b7b3-b8ab2554dbb8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c47172a0a15e409982894022d75d47ea'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:38:13.490993 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-dce4e516-cc90-41ac-b7b3-b8ab2554dbb8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c47172a0a15e409982894022d75d47ea'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:38:13.507880 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1031/2053] 10.222.0.33 () {58 vars in 1223 bytes} [Mon Jul 25 05:38:13 2022] DELETE /identity/v3/users/c47172a0a15e409982894022d75d47ea => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 25 05:38:13.533014 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-752448dd-1936-4051-be8f-0b55b2f941df admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6cdb3b365a0f44739462f84d5ab54979'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:38:13.544691 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-752448dd-1936-4051-be8f-0b55b2f941df admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6cdb3b365a0f44739462f84d5ab54979'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:38:13.545444 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-752448dd-1936-4051-be8f-0b55b2f941df admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6cdb3b365a0f44739462f84d5ab54979'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:38:13.552752 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-752448dd-1936-4051-be8f-0b55b2f941df admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6cdb3b365a0f44739462f84d5ab54979'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:38:13.553528 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-752448dd-1936-4051-be8f-0b55b2f941df admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6cdb3b365a0f44739462f84d5ab54979'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:38:13.566953 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1023/2054] 10.222.0.33 () {58 vars in 1223 bytes} [Mon Jul 25 05:38:13 2022] DELETE /identity/v3/users/6cdb3b365a0f44739462f84d5ab54979 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 25 05:38:13.859992 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce4e516-cc90-41ac-b7b3-b8ab2554dbb8 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:38:13.861510 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-dce4e516-cc90-41ac-b7b3-b8ab2554dbb8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:38:13.900372 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:DELETE=11,keystone:SELECT=88,keystone:INSERT=1 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:38:13.952431 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce4e516-cc90-41ac-b7b3-b8ab2554dbb8 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:38:13.989565 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-752448dd-1936-4051-be8f-0b55b2f941df admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:38:13.991406 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-752448dd-1936-4051-be8f-0b55b2f941df admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:38:14.045102 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a7c4b68-0037-4731-978d-6c16e1aece7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:38:14.046412 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7c4b68-0037-4731-978d-6c16e1aece7e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:38:14.046705 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7c4b68-0037-4731-978d-6c16e1aece7e admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:38:14.047299 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7c4b68-0037-4731-978d-6c16e1aece7e admin admin] PATH_INFO: `/v3/projects/c9fe142be7cc41bd91e806c2455c62d8` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:38:14.049742 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a7c4b68-0037-4731-978d-6c16e1aece7e admin admin] RBAC: Authorizing `identity:delete_project(project_id=c9fe142be7cc41bd91e806c2455c62d8)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:38:14.052484 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:38:14.052755 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a7c4b68-0037-4731-978d-6c16e1aece7e admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:38:14.072192 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-752448dd-1936-4051-be8f-0b55b2f941df admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:38:14.077093 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d943ba85-72e4-4528-88af-409ab3fffce7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:38:14.077966 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d943ba85-72e4-4528-88af-409ab3fffce7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:38:14.078256 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d943ba85-72e4-4528-88af-409ab3fffce7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:38:14.078537 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d943ba85-72e4-4528-88af-409ab3fffce7 admin admin] PATH_INFO: `/v3/projects/3f397ce61ef540909aeda972c80a34b7` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:38:14.079727 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d943ba85-72e4-4528-88af-409ab3fffce7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3f397ce61ef540909aeda972c80a34b7)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:38:14.081947 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d943ba85-72e4-4528-88af-409ab3fffce7 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:38:14.104103 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-1a7c4b68-0037-4731-978d-6c16e1aece7e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c9fe142be7cc41bd91e806c2455c62d8', 'request_id': 'req-1a7c4b68-0037-4731-978d-6c16e1aece7e'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:38:14.111246 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-1a7c4b68-0037-4731-978d-6c16e1aece7e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c9fe142be7cc41bd91e806c2455c62d8', 'request_id': 'req-1a7c4b68-0037-4731-978d-6c16e1aece7e'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:38:14.117472 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-1a7c4b68-0037-4731-978d-6c16e1aece7e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c9fe142be7cc41bd91e806c2455c62d8', 'request_id': 'req-1a7c4b68-0037-4731-978d-6c16e1aece7e'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:38:14.132309 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-1a7c4b68-0037-4731-978d-6c16e1aece7e admin admin] The token cache is being invalidate because project c9fe142be7cc41bd91e806c2455c62d8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80249) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 05:38:14.132606 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-1a7c4b68-0037-4731-978d-6c16e1aece7e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:38:14.133397 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-1a7c4b68-0037-4731-978d-6c16e1aece7e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:38:14.134954 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1032/2055] 10.222.0.33 () {58 vars in 1232 bytes} [Mon Jul 25 05:38:13 2022] DELETE /identity/v3/projects/c9fe142be7cc41bd91e806c2455c62d8 => generated 0 bytes in 275 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 05:38:14.140985 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45053e08-0df7-423c-b6dd-105f1817d796 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:38:14.141287 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45053e08-0df7-423c-b6dd-105f1817d796 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:38:14.141636 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45053e08-0df7-423c-b6dd-105f1817d796 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:38:14.148363 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-d943ba85-72e4-4528-88af-409ab3fffce7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3f397ce61ef540909aeda972c80a34b7', 'request_id': 'req-d943ba85-72e4-4528-88af-409ab3fffce7'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:38:14.155340 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-d943ba85-72e4-4528-88af-409ab3fffce7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3f397ce61ef540909aeda972c80a34b7', 'request_id': 'req-d943ba85-72e4-4528-88af-409ab3fffce7'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:38:14.160465 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-45053e08-0df7-423c-b6dd-105f1817d796 None None] MFA Rules not processed for user `1eae540242d94c10b0ad33960a855f08`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:38:14.161577 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-d943ba85-72e4-4528-88af-409ab3fffce7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3f397ce61ef540909aeda972c80a34b7', 'request_id': 'req-d943ba85-72e4-4528-88af-409ab3fffce7'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:38:14.161708 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-45053e08-0df7-423c-b6dd-105f1817d796 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:38:14.175786 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-d943ba85-72e4-4528-88af-409ab3fffce7 admin admin] The token cache is being invalidate because project 3f397ce61ef540909aeda972c80a34b7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80248) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 05:38:14.176105 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-d943ba85-72e4-4528-88af-409ab3fffce7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:38:14.176726 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-d943ba85-72e4-4528-88af-409ab3fffce7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:38:14.180850 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1024/2056] 10.222.0.33 () {58 vars in 1232 bytes} [Mon Jul 25 05:38:13 2022] DELETE /identity/v3/projects/3f397ce61ef540909aeda972c80a34b7 => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 25 05:38:14.184618 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dab9ea1-81d4-459c-8dc8-d302ccd4423a None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:38:14.185105 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dab9ea1-81d4-459c-8dc8-d302ccd4423a None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:38:14.185462 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dab9ea1-81d4-459c-8dc8-d302ccd4423a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:38:14.203794 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-2dab9ea1-81d4-459c-8dc8-d302ccd4423a None None] MFA Rules not processed for user `1eae540242d94c10b0ad33960a855f08`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:38:14.205065 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-2dab9ea1-81d4-459c-8dc8-d302ccd4423a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:38:14.214255 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:38:14.226682 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1033/2057] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 25 05:38:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 05:38:14.234087 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c308ab1-7d9f-4eb7-b3ed-2ccc7d87aa00 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:38:14.234391 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c308ab1-7d9f-4eb7-b3ed-2ccc7d87aa00 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:38:14.234676 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c308ab1-7d9f-4eb7-b3ed-2ccc7d87aa00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:38:14.252474 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-1c308ab1-7d9f-4eb7-b3ed-2ccc7d87aa00 None None] MFA Rules not processed for user `1eae540242d94c10b0ad33960a855f08`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:38:14.253771 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-1c308ab1-7d9f-4eb7-b3ed-2ccc7d87aa00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:38:14.261578 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1034/2058] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 25 05:38:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 05:38:14.266280 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c308ab1-7d9f-4eb7-b3ed-2ccc7d87aa00 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:38:14.268233 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1025/2059] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 25 05:38:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:38:14.276129 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c7eb70-5433-4300-aa7f-479d98343455 None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:38:14.276420 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c7eb70-5433-4300-aa7f-479d98343455 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:38:14.276700 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c7eb70-5433-4300-aa7f-479d98343455 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:38:14.277337 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c308ab1-7d9f-4eb7-b3ed-2ccc7d87aa00 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:38:14.282645 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d7868b-d768-4ce6-8ea1-86ce09a407cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:38:14.283234 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d7868b-d768-4ce6-8ea1-86ce09a407cf admin admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:38:14.283521 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d7868b-d768-4ce6-8ea1-86ce09a407cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:38:14.283795 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d7868b-d768-4ce6-8ea1-86ce09a407cf admin admin] PATH_INFO: `/v3/domains` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:38:14.284525 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21d7868b-d768-4ce6-8ea1-86ce09a407cf admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:38:14.284857 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21d7868b-d768-4ce6-8ea1-86ce09a407cf admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:38:14.286465 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21d7868b-d768-4ce6-8ea1-86ce09a407cf admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:38:14.295064 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-49c7eb70-5433-4300-aa7f-479d98343455 None None] MFA Rules not processed for user `1eae540242d94c10b0ad33960a855f08`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:38:14.296274 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-49c7eb70-5433-4300-aa7f-479d98343455 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:38:14.301565 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1035/2060] 10.222.0.33 () {58 vars in 1152 bytes} [Mon Jul 25 05:38:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 05:38:14.302767 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d7868b-d768-4ce6-8ea1-86ce09a407cf admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:38:14.304267 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1026/2061] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 25 05:38:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 05:38:14.308098 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c7eb70-5433-4300-aa7f-479d98343455 None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:38:14.312588 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d7868b-d768-4ce6-8ea1-86ce09a407cf admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:38:14.316914 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c7eb70-5433-4300-aa7f-479d98343455 None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:38:14.317223 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb3680fa-71d3-4d25-aece-20e4b328eb56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:38:14.317862 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3680fa-71d3-4d25-aece-20e4b328eb56 admin admin] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:38:14.318143 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3680fa-71d3-4d25-aece-20e4b328eb56 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:38:14.318423 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3680fa-71d3-4d25-aece-20e4b328eb56 admin admin] PATH_INFO: `/v3/projects` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:38:14.319108 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb3680fa-71d3-4d25-aece-20e4b328eb56 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:38:14.320997 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb3680fa-71d3-4d25-aece-20e4b328eb56 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:38:14.321724 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff3b082-814a-4463-ab92-59eb5cf377cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:38:14.322370 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff3b082-814a-4463-ab92-59eb5cf377cf admin admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:38:14.322653 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff3b082-814a-4463-ab92-59eb5cf377cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:38:14.322927 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff3b082-814a-4463-ab92-59eb5cf377cf admin admin] PATH_INFO: `/v3/domains` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:38:14.323653 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ff3b082-814a-4463-ab92-59eb5cf377cf admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:38:14.323963 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ff3b082-814a-4463-ab92-59eb5cf377cf admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:38:14.326170 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ff3b082-814a-4463-ab92-59eb5cf377cf admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:38:14.339202 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1027/2062] 10.222.0.33 () {58 vars in 1152 bytes} [Mon Jul 25 05:38:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 05:38:14.344416 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff3b082-814a-4463-ab92-59eb5cf377cf admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:38:14.352362 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1036/2063] 10.222.0.33 () {60 vars in 1152 bytes} [Mon Jul 25 05:38:14 2022] POST /identity/v3/projects => generated 369 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 05:38:14.353235 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb3680fa-71d3-4d25-aece-20e4b328eb56 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:38:14.354819 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff3b082-814a-4463-ab92-59eb5cf377cf admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:38:14.406903 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:38:14.413927 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7826f308-ab05-42f3-82d4-1bee1077fb43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:38:14.414603 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7826f308-ab05-42f3-82d4-1bee1077fb43 admin admin] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:38:14.415337 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7826f308-ab05-42f3-82d4-1bee1077fb43 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:38:14.415622 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7826f308-ab05-42f3-82d4-1bee1077fb43 admin admin] PATH_INFO: `/v3/projects` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:38:14.416586 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7826f308-ab05-42f3-82d4-1bee1077fb43 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:38:14.418199 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb3680fa-71d3-4d25-aece-20e4b328eb56 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:38:14.419641 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7826f308-ab05-42f3-82d4-1bee1077fb43 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:38:14.423552 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3814bc7-896c-4ef3-80f4-72b42a28c087 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:38:14.424250 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3814bc7-896c-4ef3-80f4-72b42a28c087 admin admin] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:38:14.424536 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3814bc7-896c-4ef3-80f4-72b42a28c087 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:38:14.424838 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3814bc7-896c-4ef3-80f4-72b42a28c087 admin admin] PATH_INFO: `/v3/users` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:38:14.425536 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3814bc7-896c-4ef3-80f4-72b42a28c087 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:38:14.427432 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3814bc7-896c-4ef3-80f4-72b42a28c087 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:38:14.446833 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1037/2064] 10.222.0.33 () {60 vars in 1143 bytes} [Mon Jul 25 05:38:14 2022] POST /identity/v3/users => generated 285 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 05:38:14.450642 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1028/2065] 10.222.0.33 () {60 vars in 1152 bytes} [Mon Jul 25 05:38:14 2022] POST /identity/v3/projects => generated 385 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 25 05:38:14.451411 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3814bc7-896c-4ef3-80f4-72b42a28c087 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:38:14.451572 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7826f308-ab05-42f3-82d4-1bee1077fb43 admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:38:14.461265 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3814bc7-896c-4ef3-80f4-72b42a28c087 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:38:14.515080 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7826f308-ab05-42f3-82d4-1bee1077fb43 admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:38:14.519262 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c143c0d-69d3-4539-ad44-0413b0e99413 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:38:14.519741 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b41b905-8359-40ca-8964-887bd72aaad0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:38:14.519934 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c143c0d-69d3-4539-ad44-0413b0e99413 admin admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:38:14.520217 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c143c0d-69d3-4539-ad44-0413b0e99413 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:38:14.520495 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c143c0d-69d3-4539-ad44-0413b0e99413 admin admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:38:14.520730 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b41b905-8359-40ca-8964-887bd72aaad0 admin admin] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:38:14.521057 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b41b905-8359-40ca-8964-887bd72aaad0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:38:14.521257 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c143c0d-69d3-4539-ad44-0413b0e99413 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:38:14.521375 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b41b905-8359-40ca-8964-887bd72aaad0 admin admin] PATH_INFO: `/v3/users` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:38:14.523892 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b41b905-8359-40ca-8964-887bd72aaad0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:38:14.524000 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c143c0d-69d3-4539-ad44-0413b0e99413 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:38:14.524155 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b41b905-8359-40ca-8964-887bd72aaad0 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:38:14.534680 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1038/2066] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jul 25 05:38:14 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 25 05:38:14.538942 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c143c0d-69d3-4539-ad44-0413b0e99413 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:38:14.545256 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1029/2067] 10.222.0.33 () {60 vars in 1143 bytes} [Mon Jul 25 05:38:14 2022] POST /identity/v3/users => generated 293 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 25 05:38:14.549233 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b41b905-8359-40ca-8964-887bd72aaad0 admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:38:14.549853 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c143c0d-69d3-4539-ad44-0413b0e99413 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:38:14.554704 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f78708-3680-42ba-bfa5-d84d28610ed9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:38:14.555370 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f78708-3680-42ba-bfa5-d84d28610ed9 admin admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:38:14.555674 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f78708-3680-42ba-bfa5-d84d28610ed9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:38:14.555937 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f78708-3680-42ba-bfa5-d84d28610ed9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:38:14.556670 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44f78708-3680-42ba-bfa5-d84d28610ed9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:38:14.558426 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44f78708-3680-42ba-bfa5-d84d28610ed9 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:38:14.558826 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b41b905-8359-40ca-8964-887bd72aaad0 admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:38:14.563664 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b061d3a9-773a-47e0-a843-cbfe1533b7f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:38:14.564324 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b061d3a9-773a-47e0-a843-cbfe1533b7f8 admin admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:38:14.564611 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b061d3a9-773a-47e0-a843-cbfe1533b7f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:38:14.564926 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b061d3a9-773a-47e0-a843-cbfe1533b7f8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:38:14.565660 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b061d3a9-773a-47e0-a843-cbfe1533b7f8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:38:14.567876 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b061d3a9-773a-47e0-a843-cbfe1533b7f8 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:38:14.568957 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1039/2068] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jul 25 05:38:14 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 25 05:38:14.572942 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f78708-3680-42ba-bfa5-d84d28610ed9 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:38:14.579389 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1030/2069] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jul 25 05:38:14 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 05:38:14.583416 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b061d3a9-773a-47e0-a843-cbfe1533b7f8 admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:38:14.586184 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f78708-3680-42ba-bfa5-d84d28610ed9 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:38:14.592096 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9c5bfb-2492-40db-aff2-43d6d31d79ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:38:14.593031 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9c5bfb-2492-40db-aff2-43d6d31d79ce admin admin] REQUEST_METHOD: `PUT` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:38:14.593315 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9c5bfb-2492-40db-aff2-43d6d31d79ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:38:14.593593 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9c5bfb-2492-40db-aff2-43d6d31d79ce admin admin] PATH_INFO: `/v3/projects/c57e21c048594eed811473557c7f5694/users/44f81ab87d404910aca84404a53579dc/roles/5074e74e35f54d4b9bb415c34999a82a` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:38:14.594234 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b061d3a9-773a-47e0-a843-cbfe1533b7f8 admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:38:14.598817 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca1496fb-42c4-44ce-943b-b0b41eae24f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:38:14.599511 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1496fb-42c4-44ce-943b-b0b41eae24f0 admin admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:38:14.599865 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1496fb-42c4-44ce-943b-b0b41eae24f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:38:14.600209 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1496fb-42c4-44ce-943b-b0b41eae24f0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:38:14.601033 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca1496fb-42c4-44ce-943b-b0b41eae24f0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:38:14.602671 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca1496fb-42c4-44ce-943b-b0b41eae24f0 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:38:14.608949 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a9c5bfb-2492-40db-aff2-43d6d31d79ce admin admin] RBAC: Authorizing `identity:create_grant(project_id=c57e21c048594eed811473557c7f5694, user_id=44f81ab87d404910aca84404a53579dc, role_id=5074e74e35f54d4b9bb415c34999a82a)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:38:14.611596 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a9c5bfb-2492-40db-aff2-43d6d31d79ce admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:38:14.613322 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1031/2070] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jul 25 05:38:14 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 25 05:38:14.617394 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca1496fb-42c4-44ce-943b-b0b41eae24f0 admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:38:14.626253 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1040/2071] 10.222.0.33 () {60 vars in 1482 bytes} [Mon Jul 25 05:38:14 2022] PUT /identity/v3/projects/c57e21c048594eed811473557c7f5694/users/44f81ab87d404910aca84404a53579dc/roles/5074e74e35f54d4b9bb415c34999a82a => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 05:38:14.627898 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca1496fb-42c4-44ce-943b-b0b41eae24f0 admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:38:14.636407 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7cad97-d6b1-444e-8673-87a68710952d None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:38:14.636702 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7cad97-d6b1-444e-8673-87a68710952d None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:38:14.637017 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7cad97-d6b1-444e-8673-87a68710952d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:38:14.649102 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:38:14.665646 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-ef7cad97-d6b1-444e-8673-87a68710952d None None] MFA Rules not processed for user `44f81ab87d404910aca84404a53579dc`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:38:14.667100 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-ef7cad97-d6b1-444e-8673-87a68710952d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:38:14.689329 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cbfdba0-b66b-45fe-b120-f40ea84f193e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:38:14.690229 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbfdba0-b66b-45fe-b120-f40ea84f193e admin admin] REQUEST_METHOD: `PUT` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:38:14.690515 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbfdba0-b66b-45fe-b120-f40ea84f193e admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:38:14.690795 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbfdba0-b66b-45fe-b120-f40ea84f193e admin admin] PATH_INFO: `/v3/projects/2685c7c4b408488f9037dceb15f2300c/users/1537bbc6582f427aa1ef0f982db53125/roles/5074e74e35f54d4b9bb415c34999a82a` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:38:14.706340 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cbfdba0-b66b-45fe-b120-f40ea84f193e admin admin] RBAC: Authorizing `identity:create_grant(project_id=2685c7c4b408488f9037dceb15f2300c, user_id=1537bbc6582f427aa1ef0f982db53125, role_id=5074e74e35f54d4b9bb415c34999a82a)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:38:14.709043 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cbfdba0-b66b-45fe-b120-f40ea84f193e admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:38:14.722814 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1032/2072] 10.222.0.33 () {60 vars in 1482 bytes} [Mon Jul 25 05:38:14 2022] PUT /identity/v3/projects/2685c7c4b408488f9037dceb15f2300c/users/1537bbc6582f427aa1ef0f982db53125/roles/5074e74e35f54d4b9bb415c34999a82a => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 25 05:38:14.735365 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba041f81-cc32-4970-b41f-07eb05caf5f3 None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:38:14.735660 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba041f81-cc32-4970-b41f-07eb05caf5f3 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:38:14.735941 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba041f81-cc32-4970-b41f-07eb05caf5f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:38:14.736315 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1041/2073] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 25 05:38:14 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 05:38:14.748877 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef7cad97-d6b1-444e-8673-87a68710952d None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:38:14.750268 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-ef7cad97-d6b1-444e-8673-87a68710952d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:38:14.762683 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-ba041f81-cc32-4970-b41f-07eb05caf5f3 None None] MFA Rules not processed for user `1537bbc6582f427aa1ef0f982db53125`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:38:14.764004 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-ba041f81-cc32-4970-b41f-07eb05caf5f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:38:14.784808 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:38:14.838064 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1033/2074] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 25 05:38:14 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 05:38:14.850436 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba041f81-cc32-4970-b41f-07eb05caf5f3 None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:38:14.852643 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-ba041f81-cc32-4970-b41f-07eb05caf5f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:38:14.854496 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef7cad97-d6b1-444e-8673-87a68710952d None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:38:14.928428 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b5f1a27-40b6-40bd-b7f4-eca12400500d service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de99e44bdf9c4030adbe9c0efa3831cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92afba266cce4d1bad4134c351e74be3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:38:14.929129 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5f1a27-40b6-40bd-b7f4-eca12400500d service glance] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:38:14.929419 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5f1a27-40b6-40bd-b7f4-eca12400500d service glance] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:38:14.929703 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5f1a27-40b6-40bd-b7f4-eca12400500d service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:38:14.930511 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba041f81-cc32-4970-b41f-07eb05caf5f3 None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:38:14.935355 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f7873b-04aa-45e7-a298-64c73b3c7030 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db57b478c25643dbaf936f90f6e17d0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92afba266cce4d1bad4134c351e74be3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:38:14.936114 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f7873b-04aa-45e7-a298-64c73b3c7030 service cinder] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:38:14.936460 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f7873b-04aa-45e7-a298-64c73b3c7030 service cinder] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:38:14.937451 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f7873b-04aa-45e7-a298-64c73b3c7030 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:38:14.948852 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4f7873b-04aa-45e7-a298-64c73b3c7030 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:38:14.950826 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4f7873b-04aa-45e7-a298-64c73b3c7030 service cinder] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:38:14.960798 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1034/2075] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jul 25 05:38:14 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 05:38:14.982025 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f7873b-04aa-45e7-a298-64c73b3c7030 service cinder] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:38:14.990965 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:38:15.005762 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b5f1a27-40b6-40bd-b7f4-eca12400500d service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:38:15.007214 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b5f1a27-40b6-40bd-b7f4-eca12400500d service glance] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:38:15.021658 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1042/2076] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jul 25 05:38:14 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 273 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 25 05:38:15.050750 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f7873b-04aa-45e7-a298-64c73b3c7030 service cinder] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:38:15.055894 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc8cac3-37f3-4ed8-bb6e-bfe9bb58985b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:38:15.056773 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc8cac3-37f3-4ed8-bb6e-bfe9bb58985b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:38:15.057214 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc8cac3-37f3-4ed8-bb6e-bfe9bb58985b admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:38:15.057489 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc8cac3-37f3-4ed8-bb6e-bfe9bb58985b admin admin] PATH_INFO: `/v3/users/1537bbc6582f427aa1ef0f982db53125` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:38:15.058707 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfc8cac3-37f3-4ed8-bb6e-bfe9bb58985b admin admin] RBAC: Authorizing `identity:delete_user(user_id=1537bbc6582f427aa1ef0f982db53125)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:38:15.061447 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfc8cac3-37f3-4ed8-bb6e-bfe9bb58985b admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:38:15.121435 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-bfc8cac3-37f3-4ed8-bb6e-bfe9bb58985b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1537bbc6582f427aa1ef0f982db53125'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:38:15.138643 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-bfc8cac3-37f3-4ed8-bb6e-bfe9bb58985b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1537bbc6582f427aa1ef0f982db53125'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:38:15.139711 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-bfc8cac3-37f3-4ed8-bb6e-bfe9bb58985b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1537bbc6582f427aa1ef0f982db53125'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:38:15.143529 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:38:15.151280 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-bfc8cac3-37f3-4ed8-bb6e-bfe9bb58985b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1537bbc6582f427aa1ef0f982db53125'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:38:15.152159 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-bfc8cac3-37f3-4ed8-bb6e-bfe9bb58985b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1537bbc6582f427aa1ef0f982db53125'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:38:15.170626 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1035/2077] 10.222.0.33 () {58 vars in 1223 bytes} [Mon Jul 25 05:38:14 2022] DELETE /identity/v3/users/1537bbc6582f427aa1ef0f982db53125 => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 05:38:15.175249 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f028385-95f4-4e96-8917-4b0e1ee34a87 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:38:15.175722 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f028385-95f4-4e96-8917-4b0e1ee34a87 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:38:15.175863 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f028385-95f4-4e96-8917-4b0e1ee34a87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:38:15.203010 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-4f028385-95f4-4e96-8917-4b0e1ee34a87 None None] MFA Rules not processed for user `1eae540242d94c10b0ad33960a855f08`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:38:15.204547 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-4f028385-95f4-4e96-8917-4b0e1ee34a87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:38:15.268456 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1043/2078] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 25 05:38:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 05:38:15.279602 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc8cac3-37f3-4ed8-bb6e-bfe9bb58985b admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:38:15.281612 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-bfc8cac3-37f3-4ed8-bb6e-bfe9bb58985b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:38:15.343236 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc8cac3-37f3-4ed8-bb6e-bfe9bb58985b admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:38:15.347533 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b05e2dcf-70db-4a57-9369-54b25c7de582 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b2931200f99e43d79a50a9f6e02738e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92afba266cce4d1bad4134c351e74be3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:38:15.348223 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05e2dcf-70db-4a57-9369-54b25c7de582 service neutron] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:38:15.348539 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05e2dcf-70db-4a57-9369-54b25c7de582 service neutron] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:38:15.348846 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05e2dcf-70db-4a57-9369-54b25c7de582 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:38:15.364911 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b05e2dcf-70db-4a57-9369-54b25c7de582 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:38:15.366681 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b05e2dcf-70db-4a57-9369-54b25c7de582 service neutron] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:38:15.382056 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1036/2079] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jul 25 05:38:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 05:38:15.687083 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f028385-95f4-4e96-8917-4b0e1ee34a87 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:38:15.688595 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-4f028385-95f4-4e96-8917-4b0e1ee34a87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:38:15.702211 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f028385-95f4-4e96-8917-4b0e1ee34a87 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:38:15.707153 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca82545-8707-489c-b3dc-eaffc4e6ccd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:38:15.708014 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca82545-8707-489c-b3dc-eaffc4e6ccd2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:38:15.708298 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca82545-8707-489c-b3dc-eaffc4e6ccd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:38:15.708574 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca82545-8707-489c-b3dc-eaffc4e6ccd2 admin admin] PATH_INFO: `/v3/projects/2685c7c4b408488f9037dceb15f2300c` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:38:15.709790 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ca82545-8707-489c-b3dc-eaffc4e6ccd2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2685c7c4b408488f9037dceb15f2300c)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:38:15.712392 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ca82545-8707-489c-b3dc-eaffc4e6ccd2 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:38:15.759861 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-7ca82545-8707-489c-b3dc-eaffc4e6ccd2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2685c7c4b408488f9037dceb15f2300c', 'request_id': 'req-7ca82545-8707-489c-b3dc-eaffc4e6ccd2'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:38:15.765775 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-7ca82545-8707-489c-b3dc-eaffc4e6ccd2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2685c7c4b408488f9037dceb15f2300c', 'request_id': 'req-7ca82545-8707-489c-b3dc-eaffc4e6ccd2'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:38:15.772034 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-7ca82545-8707-489c-b3dc-eaffc4e6ccd2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2685c7c4b408488f9037dceb15f2300c', 'request_id': 'req-7ca82545-8707-489c-b3dc-eaffc4e6ccd2'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:38:15.785478 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-7ca82545-8707-489c-b3dc-eaffc4e6ccd2 admin admin] The token cache is being invalidate because project 2685c7c4b408488f9037dceb15f2300c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80249) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 05:38:15.785773 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-7ca82545-8707-489c-b3dc-eaffc4e6ccd2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:38:15.786397 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-7ca82545-8707-489c-b3dc-eaffc4e6ccd2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:38:15.787954 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1044/2080] 10.222.0.33 () {58 vars in 1232 bytes} [Mon Jul 25 05:38:15 2022] DELETE /identity/v3/projects/2685c7c4b408488f9037dceb15f2300c => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 05:38:15.792958 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f452aa69-8271-43db-8861-9ed04468b8be None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:38:15.793249 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f452aa69-8271-43db-8861-9ed04468b8be None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:38:15.793591 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f452aa69-8271-43db-8861-9ed04468b8be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:38:15.811572 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-f452aa69-8271-43db-8861-9ed04468b8be None None] MFA Rules not processed for user `1eae540242d94c10b0ad33960a855f08`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:38:15.812761 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-f452aa69-8271-43db-8861-9ed04468b8be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:38:15.878217 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1037/2081] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 25 05:38:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 05:38:15.886601 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05bf3d90-b2ff-406b-b6dd-cb37eb696fe6 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:38:15.888048 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05bf3d90-b2ff-406b-b6dd-cb37eb696fe6 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:38:15.889525 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05bf3d90-b2ff-406b-b6dd-cb37eb696fe6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:38:15.913020 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-05bf3d90-b2ff-406b-b6dd-cb37eb696fe6 None None] MFA Rules not processed for user `1eae540242d94c10b0ad33960a855f08`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:38:15.914375 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-05bf3d90-b2ff-406b-b6dd-cb37eb696fe6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:38:15.926530 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1045/2082] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 25 05:38:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:38:15.931228 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f452aa69-8271-43db-8861-9ed04468b8be None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:38:15.942525 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f452aa69-8271-43db-8861-9ed04468b8be None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:38:15.947308 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719315a6-2f64-422f-9c55-2a0ac07839a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:38:15.948045 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719315a6-2f64-422f-9c55-2a0ac07839a8 admin admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:38:15.948492 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719315a6-2f64-422f-9c55-2a0ac07839a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:38:15.948857 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719315a6-2f64-422f-9c55-2a0ac07839a8 admin admin] PATH_INFO: `/v3/domains` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:38:15.949512 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-719315a6-2f64-422f-9c55-2a0ac07839a8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:38:15.950942 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-719315a6-2f64-422f-9c55-2a0ac07839a8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:38:15.952565 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-719315a6-2f64-422f-9c55-2a0ac07839a8 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:38:15.976859 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05bf3d90-b2ff-406b-b6dd-cb37eb696fe6 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:38:15.984967 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1038/2083] 10.222.0.33 () {58 vars in 1152 bytes} [Mon Jul 25 05:38:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:38:15.988568 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05bf3d90-b2ff-406b-b6dd-cb37eb696fe6 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:38:15.993436 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49216616-087d-4551-9023-3743d0738afe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:38:15.994103 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49216616-087d-4551-9023-3743d0738afe admin admin] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:38:15.994388 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49216616-087d-4551-9023-3743d0738afe admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:38:15.994663 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49216616-087d-4551-9023-3743d0738afe admin admin] PATH_INFO: `/v3/projects` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:38:15.995355 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49216616-087d-4551-9023-3743d0738afe admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:38:15.997756 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49216616-087d-4551-9023-3743d0738afe admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:38:16.032557 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1046/2084] 10.222.0.33 () {60 vars in 1152 bytes} [Mon Jul 25 05:38:15 2022] POST /identity/v3/projects => generated 361 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 05:38:16.036697 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719315a6-2f64-422f-9c55-2a0ac07839a8 admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:38:16.048101 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719315a6-2f64-422f-9c55-2a0ac07839a8 admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:38:16.128575 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9835a240-c774-4160-ad99-2222ba6ba671 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:38:16.129411 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9835a240-c774-4160-ad99-2222ba6ba671 admin admin] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:38:16.130178 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9835a240-c774-4160-ad99-2222ba6ba671 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:38:16.130178 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9835a240-c774-4160-ad99-2222ba6ba671 admin admin] PATH_INFO: `/v3/users` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:38:16.130543 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9835a240-c774-4160-ad99-2222ba6ba671 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:38:16.132488 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9835a240-c774-4160-ad99-2222ba6ba671 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:38:16.163624 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1039/2085] 10.222.0.33 () {60 vars in 1143 bytes} [Mon Jul 25 05:38:16 2022] POST /identity/v3/users => generated 281 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 05:38:16.164048 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49216616-087d-4551-9023-3743d0738afe admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:38:16.179417 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49216616-087d-4551-9023-3743d0738afe admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:38:16.201952 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-310f5e3f-4efb-4469-9592-eb246ad46981 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:38:16.202653 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310f5e3f-4efb-4469-9592-eb246ad46981 admin admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:38:16.202946 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310f5e3f-4efb-4469-9592-eb246ad46981 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:38:16.203256 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310f5e3f-4efb-4469-9592-eb246ad46981 admin admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:38:16.203993 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-310f5e3f-4efb-4469-9592-eb246ad46981 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:38:16.206094 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-310f5e3f-4efb-4469-9592-eb246ad46981 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:38:16.229085 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1047/2086] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jul 25 05:38:16 2022] GET /identity/v3/roles => generated 1420 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 05:38:16.230412 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9835a240-c774-4160-ad99-2222ba6ba671 admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:38:16.238157 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:DELETE=1,keystone:SELECT=96,keystone:INSERT=3 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:38:16.242210 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9835a240-c774-4160-ad99-2222ba6ba671 admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:38:16.254476 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e54ad77-43c4-4393-9d4f-3ba09e15ba33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:38:16.255054 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e54ad77-43c4-4393-9d4f-3ba09e15ba33 admin admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:38:16.256468 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e54ad77-43c4-4393-9d4f-3ba09e15ba33 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:38:16.256468 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e54ad77-43c4-4393-9d4f-3ba09e15ba33 admin admin] PATH_INFO: `/v3/roles` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:38:16.257289 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e54ad77-43c4-4393-9d4f-3ba09e15ba33 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:38:16.260085 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e54ad77-43c4-4393-9d4f-3ba09e15ba33 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:38:16.285584 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1040/2087] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jul 25 05:38:16 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 25 05:38:16.289994 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-310f5e3f-4efb-4469-9592-eb246ad46981 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:38:16.306651 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-310f5e3f-4efb-4469-9592-eb246ad46981 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:38:16.313447 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d36985a1-2514-4f8f-a9a2-04799266c189 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:38:16.314455 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36985a1-2514-4f8f-a9a2-04799266c189 admin admin] REQUEST_METHOD: `PUT` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:38:16.314818 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36985a1-2514-4f8f-a9a2-04799266c189 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:38:16.315171 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36985a1-2514-4f8f-a9a2-04799266c189 admin admin] PATH_INFO: `/v3/projects/c96e02abedb04bb6b70e2df7e6a77567/users/d8b768400a5c4e0babb8f648622b69a1/roles/5074e74e35f54d4b9bb415c34999a82a` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:38:16.333644 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d36985a1-2514-4f8f-a9a2-04799266c189 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c96e02abedb04bb6b70e2df7e6a77567, user_id=d8b768400a5c4e0babb8f648622b69a1, role_id=5074e74e35f54d4b9bb415c34999a82a)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:38:16.337740 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d36985a1-2514-4f8f-a9a2-04799266c189 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:38:16.359122 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1048/2088] 10.222.0.33 () {60 vars in 1482 bytes} [Mon Jul 25 05:38:16 2022] PUT /identity/v3/projects/c96e02abedb04bb6b70e2df7e6a77567/users/d8b768400a5c4e0babb8f648622b69a1/roles/5074e74e35f54d4b9bb415c34999a82a => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 05:38:16.370285 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40ca8d7-f8d2-45df-812f-909f1fec8758 None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:38:16.370582 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40ca8d7-f8d2-45df-812f-909f1fec8758 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:38:16.370964 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40ca8d7-f8d2-45df-812f-909f1fec8758 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:38:16.401077 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-d40ca8d7-f8d2-45df-812f-909f1fec8758 None None] MFA Rules not processed for user `d8b768400a5c4e0babb8f648622b69a1`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:38:16.402522 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-d40ca8d7-f8d2-45df-812f-909f1fec8758 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:38:16.493313 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1041/2089] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 25 05:38:16 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:38:16.529716 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d36985a1-2514-4f8f-a9a2-04799266c189 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:38:16.531487 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-d36985a1-2514-4f8f-a9a2-04799266c189 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:38:16.542024 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:38:16.617963 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d36985a1-2514-4f8f-a9a2-04799266c189 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:38:16.622866 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b66d0b36-8eb1-4074-914a-3cb0a9353f80 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db57b478c25643dbaf936f90f6e17d0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92afba266cce4d1bad4134c351e74be3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:38:16.623967 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66d0b36-8eb1-4074-914a-3cb0a9353f80 service cinder] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:38:16.624554 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66d0b36-8eb1-4074-914a-3cb0a9353f80 service cinder] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:38:16.624934 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66d0b36-8eb1-4074-914a-3cb0a9353f80 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:38:16.640631 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b66d0b36-8eb1-4074-914a-3cb0a9353f80 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:38:16.642730 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b66d0b36-8eb1-4074-914a-3cb0a9353f80 service cinder] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:38:16.656903 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1049/2090] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jul 25 05:38:16 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 05:38:16.668132 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d40ca8d7-f8d2-45df-812f-909f1fec8758 None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:38:16.750986 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d40ca8d7-f8d2-45df-812f-909f1fec8758 None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:38:16.755620 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9325eb05-6383-47d5-b9b0-87990b310a43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:38:16.756679 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9325eb05-6383-47d5-b9b0-87990b310a43 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:38:16.758002 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9325eb05-6383-47d5-b9b0-87990b310a43 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:38:16.759263 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9325eb05-6383-47d5-b9b0-87990b310a43 admin admin] PATH_INFO: `/v3/users/d8b768400a5c4e0babb8f648622b69a1` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:38:16.761347 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9325eb05-6383-47d5-b9b0-87990b310a43 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d8b768400a5c4e0babb8f648622b69a1)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:38:16.762910 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9325eb05-6383-47d5-b9b0-87990b310a43 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:38:16.821708 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-9325eb05-6383-47d5-b9b0-87990b310a43 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd8b768400a5c4e0babb8f648622b69a1'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:38:16.839385 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-9325eb05-6383-47d5-b9b0-87990b310a43 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd8b768400a5c4e0babb8f648622b69a1'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:38:16.840150 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-9325eb05-6383-47d5-b9b0-87990b310a43 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd8b768400a5c4e0babb8f648622b69a1'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:38:16.853039 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-9325eb05-6383-47d5-b9b0-87990b310a43 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd8b768400a5c4e0babb8f648622b69a1'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:38:16.853995 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-9325eb05-6383-47d5-b9b0-87990b310a43 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd8b768400a5c4e0babb8f648622b69a1'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:38:16.874803 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1042/2091] 10.222.0.33 () {58 vars in 1223 bytes} [Mon Jul 25 05:38:16 2022] DELETE /identity/v3/users/d8b768400a5c4e0babb8f648622b69a1 => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 25 05:38:16.876577 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469e21df-4a72-474f-af11-e6a9d55fca62 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:38:16.877501 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469e21df-4a72-474f-af11-e6a9d55fca62 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:38:16.877766 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469e21df-4a72-474f-af11-e6a9d55fca62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:38:16.908565 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-469e21df-4a72-474f-af11-e6a9d55fca62 None None] MFA Rules not processed for user `1eae540242d94c10b0ad33960a855f08`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:38:16.909855 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-469e21df-4a72-474f-af11-e6a9d55fca62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:38:16.999408 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1050/2092] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 25 05:38:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:38:17.010494 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9325eb05-6383-47d5-b9b0-87990b310a43 admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:38:17.012267 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-9325eb05-6383-47d5-b9b0-87990b310a43 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:38:17.019180 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:38:17.084952 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9325eb05-6383-47d5-b9b0-87990b310a43 admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:38:17.087604 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e27d14cf-7d69-48e6-985c-929f17e2026d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b2931200f99e43d79a50a9f6e02738e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92afba266cce4d1bad4134c351e74be3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:38:17.088259 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27d14cf-7d69-48e6-985c-929f17e2026d service neutron] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:38:17.088548 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27d14cf-7d69-48e6-985c-929f17e2026d service neutron] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:38:17.088854 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27d14cf-7d69-48e6-985c-929f17e2026d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:38:17.099046 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e27d14cf-7d69-48e6-985c-929f17e2026d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:38:17.100500 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e27d14cf-7d69-48e6-985c-929f17e2026d service neutron] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:38:17.111454 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1043/2093] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jul 25 05:38:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 05:38:17.446366 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-469e21df-4a72-474f-af11-e6a9d55fca62 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:38:17.449457 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-469e21df-4a72-474f-af11-e6a9d55fca62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:38:17.483748 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-469e21df-4a72-474f-af11-e6a9d55fca62 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:38:17.490211 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ee0090-0161-421b-be0e-3fef4eae99ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:38:17.491461 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ee0090-0161-421b-be0e-3fef4eae99ec admin admin] REQUEST_METHOD: `DELETE` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:38:17.491914 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ee0090-0161-421b-be0e-3fef4eae99ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:38:17.492347 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ee0090-0161-421b-be0e-3fef4eae99ec admin admin] PATH_INFO: `/v3/projects/c96e02abedb04bb6b70e2df7e6a77567` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:38:17.494761 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5ee0090-0161-421b-be0e-3fef4eae99ec admin admin] RBAC: Authorizing `identity:delete_project(project_id=c96e02abedb04bb6b70e2df7e6a77567)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:38:17.497115 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5ee0090-0161-421b-be0e-3fef4eae99ec admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:38:17.608937 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-d5ee0090-0161-421b-be0e-3fef4eae99ec admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c96e02abedb04bb6b70e2df7e6a77567', 'request_id': 'req-d5ee0090-0161-421b-be0e-3fef4eae99ec'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:38:17.615808 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-d5ee0090-0161-421b-be0e-3fef4eae99ec admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c96e02abedb04bb6b70e2df7e6a77567', 'request_id': 'req-d5ee0090-0161-421b-be0e-3fef4eae99ec'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:38:17.625042 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-d5ee0090-0161-421b-be0e-3fef4eae99ec admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c96e02abedb04bb6b70e2df7e6a77567', 'request_id': 'req-d5ee0090-0161-421b-be0e-3fef4eae99ec'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:38:17.654415 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-d5ee0090-0161-421b-be0e-3fef4eae99ec admin admin] The token cache is being invalidate because project c96e02abedb04bb6b70e2df7e6a77567 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80249) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 05:38:17.654922 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-d5ee0090-0161-421b-be0e-3fef4eae99ec admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:38:17.655671 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-d5ee0090-0161-421b-be0e-3fef4eae99ec admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:38:17.657798 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1051/2094] 10.222.0.33 () {58 vars in 1232 bytes} [Mon Jul 25 05:38:17 2022] DELETE /identity/v3/projects/c96e02abedb04bb6b70e2df7e6a77567 => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 05:38:17.662958 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38710ae6-e726-464b-81e4-d7de4cf7ffd3 None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:38:17.663508 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38710ae6-e726-464b-81e4-d7de4cf7ffd3 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:38:17.664037 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38710ae6-e726-464b-81e4-d7de4cf7ffd3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:38:17.686694 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-38710ae6-e726-464b-81e4-d7de4cf7ffd3 None None] MFA Rules not processed for user `1eae540242d94c10b0ad33960a855f08`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:38:17.688093 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-38710ae6-e726-464b-81e4-d7de4cf7ffd3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:38:17.792053 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1044/2095] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 25 05:38:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:38:17.802601 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8489151-46a9-4b75-bbd2-f1bc3408e0bd None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:38:17.803406 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8489151-46a9-4b75-bbd2-f1bc3408e0bd None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:38:17.803653 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8489151-46a9-4b75-bbd2-f1bc3408e0bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:38:17.862680 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-a8489151-46a9-4b75-bbd2-f1bc3408e0bd None None] MFA Rules not processed for user `1eae540242d94c10b0ad33960a855f08`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:38:17.866105 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-a8489151-46a9-4b75-bbd2-f1bc3408e0bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:38:17.882135 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1052/2096] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 25 05:38:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:38:17.886797 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38710ae6-e726-464b-81e4-d7de4cf7ffd3 None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:38:17.896695 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38710ae6-e726-464b-81e4-d7de4cf7ffd3 None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:38:17.905607 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba12336e-04e1-44c8-886a-f867dbed26f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:38:17.906221 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba12336e-04e1-44c8-886a-f867dbed26f5 admin admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:38:17.906505 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba12336e-04e1-44c8-886a-f867dbed26f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:38:17.906779 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba12336e-04e1-44c8-886a-f867dbed26f5 admin admin] PATH_INFO: `/v3/domains` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:38:17.907512 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba12336e-04e1-44c8-886a-f867dbed26f5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:38:17.907819 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba12336e-04e1-44c8-886a-f867dbed26f5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:38:17.909493 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba12336e-04e1-44c8-886a-f867dbed26f5 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:38:17.925865 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1045/2097] 10.222.0.33 () {58 vars in 1152 bytes} [Mon Jul 25 05:38:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 05:38:17.927035 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8489151-46a9-4b75-bbd2-f1bc3408e0bd None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:38:17.936575 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8489151-46a9-4b75-bbd2-f1bc3408e0bd None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:38:17.941610 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39925bf2-a882-44b4-aaca-8a5a57a7fa2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:38:17.942264 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39925bf2-a882-44b4-aaca-8a5a57a7fa2d admin admin] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:38:17.942548 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39925bf2-a882-44b4-aaca-8a5a57a7fa2d admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:38:17.942820 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39925bf2-a882-44b4-aaca-8a5a57a7fa2d admin admin] PATH_INFO: `/v3/projects` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:38:17.943556 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39925bf2-a882-44b4-aaca-8a5a57a7fa2d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:38:17.945525 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39925bf2-a882-44b4-aaca-8a5a57a7fa2d admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:38:17.972203 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1053/2098] 10.222.0.33 () {60 vars in 1152 bytes} [Mon Jul 25 05:38:17 2022] POST /identity/v3/projects => generated 375 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 05:38:17.976485 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba12336e-04e1-44c8-886a-f867dbed26f5 admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:38:17.987941 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba12336e-04e1-44c8-886a-f867dbed26f5 admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:38:18.051170 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ab8f7f-c8e3-4a6b-8e1d-6db43ec76a76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:38:18.051849 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ab8f7f-c8e3-4a6b-8e1d-6db43ec76a76 admin admin] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:38:18.052146 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ab8f7f-c8e3-4a6b-8e1d-6db43ec76a76 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:38:18.052430 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ab8f7f-c8e3-4a6b-8e1d-6db43ec76a76 admin admin] PATH_INFO: `/v3/users` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:38:18.053183 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8ab8f7f-c8e3-4a6b-8e1d-6db43ec76a76 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:38:18.055130 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8ab8f7f-c8e3-4a6b-8e1d-6db43ec76a76 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:38:18.078202 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1046/2099] 10.222.0.33 () {60 vars in 1143 bytes} [Mon Jul 25 05:38:17 2022] POST /identity/v3/users => generated 288 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 25 05:38:18.082742 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39925bf2-a882-44b4-aaca-8a5a57a7fa2d admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:38:18.089537 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:38:18.093915 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39925bf2-a882-44b4-aaca-8a5a57a7fa2d admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:38:18.099815 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d5860f-e7c1-4987-9d5d-382aa5672509 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:38:18.100597 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d5860f-e7c1-4987-9d5d-382aa5672509 admin admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:38:18.101077 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d5860f-e7c1-4987-9d5d-382aa5672509 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:38:18.101361 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d5860f-e7c1-4987-9d5d-382aa5672509 admin admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:38:18.102094 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d5860f-e7c1-4987-9d5d-382aa5672509 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:38:18.103718 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d5860f-e7c1-4987-9d5d-382aa5672509 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:38:18.115158 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1054/2100] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jul 25 05:38:18 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 05:38:18.119255 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ab8f7f-c8e3-4a6b-8e1d-6db43ec76a76 admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:38:18.131296 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ab8f7f-c8e3-4a6b-8e1d-6db43ec76a76 admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:38:18.136740 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a4dfff7-9c4f-46f9-b51e-60392cf5b5d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:38:18.137592 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4dfff7-9c4f-46f9-b51e-60392cf5b5d4 admin admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:38:18.138085 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4dfff7-9c4f-46f9-b51e-60392cf5b5d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:38:18.138229 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4dfff7-9c4f-46f9-b51e-60392cf5b5d4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:38:18.139067 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a4dfff7-9c4f-46f9-b51e-60392cf5b5d4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:38:18.140673 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a4dfff7-9c4f-46f9-b51e-60392cf5b5d4 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:38:18.153352 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1047/2101] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jul 25 05:38:18 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 05:38:18.158084 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d5860f-e7c1-4987-9d5d-382aa5672509 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:38:18.167938 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d5860f-e7c1-4987-9d5d-382aa5672509 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:38:18.173246 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f21fef22-cd40-4e19-bec9-ee7c08da65b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:38:18.174127 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21fef22-cd40-4e19-bec9-ee7c08da65b0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:38:18.174414 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21fef22-cd40-4e19-bec9-ee7c08da65b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:38:18.174694 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21fef22-cd40-4e19-bec9-ee7c08da65b0 admin admin] PATH_INFO: `/v3/projects/04e3a31c2c3c488aadcce59baecefef2/users/2ea5d6767fe3420593c6d981055b069c/roles/5074e74e35f54d4b9bb415c34999a82a` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:38:18.191179 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f21fef22-cd40-4e19-bec9-ee7c08da65b0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=04e3a31c2c3c488aadcce59baecefef2, user_id=2ea5d6767fe3420593c6d981055b069c, role_id=5074e74e35f54d4b9bb415c34999a82a)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:38:18.193889 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f21fef22-cd40-4e19-bec9-ee7c08da65b0 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:38:18.206855 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1055/2102] 10.222.0.33 () {60 vars in 1482 bytes} [Mon Jul 25 05:38:18 2022] PUT /identity/v3/projects/04e3a31c2c3c488aadcce59baecefef2/users/2ea5d6767fe3420593c6d981055b069c/roles/5074e74e35f54d4b9bb415c34999a82a => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 05:38:18.217414 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d07571-d090-4bc0-9421-2f1ff4bd8f9b None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:38:18.217708 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d07571-d090-4bc0-9421-2f1ff4bd8f9b None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:38:18.217989 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d07571-d090-4bc0-9421-2f1ff4bd8f9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:38:18.239224 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:38:18.249064 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-86d07571-d090-4bc0-9421-2f1ff4bd8f9b None None] MFA Rules not processed for user `2ea5d6767fe3420593c6d981055b069c`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:38:18.250508 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-86d07571-d090-4bc0-9421-2f1ff4bd8f9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:38:18.317134 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1048/2103] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 25 05:38:18 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:38:18.332113 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f21fef22-cd40-4e19-bec9-ee7c08da65b0 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:38:18.333853 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-f21fef22-cd40-4e19-bec9-ee7c08da65b0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:38:18.400748 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f21fef22-cd40-4e19-bec9-ee7c08da65b0 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:38:18.404743 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc2ba33-93fb-437b-b344-89f51d1a0344 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db57b478c25643dbaf936f90f6e17d0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92afba266cce4d1bad4134c351e74be3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:38:18.405441 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc2ba33-93fb-437b-b344-89f51d1a0344 service cinder] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:38:18.405737 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc2ba33-93fb-437b-b344-89f51d1a0344 service cinder] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:38:18.406021 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc2ba33-93fb-437b-b344-89f51d1a0344 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:38:18.419527 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dc2ba33-93fb-437b-b344-89f51d1a0344 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:38:18.421190 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dc2ba33-93fb-437b-b344-89f51d1a0344 service cinder] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:38:18.434723 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1056/2104] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jul 25 05:38:18 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 05:38:26.787686 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d07571-d090-4bc0-9421-2f1ff4bd8f9b None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:38:26.873684 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d07571-d090-4bc0-9421-2f1ff4bd8f9b None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:38:26.879930 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9483ae03-3fcd-41cc-8c50-be368fd9f1bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:38:26.880999 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9483ae03-3fcd-41cc-8c50-be368fd9f1bd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:38:26.881369 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9483ae03-3fcd-41cc-8c50-be368fd9f1bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:38:26.881747 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9483ae03-3fcd-41cc-8c50-be368fd9f1bd admin admin] PATH_INFO: `/v3/users/2ea5d6767fe3420593c6d981055b069c` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:38:26.883228 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9483ae03-3fcd-41cc-8c50-be368fd9f1bd admin admin] RBAC: Authorizing `identity:delete_user(user_id=2ea5d6767fe3420593c6d981055b069c)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:38:26.886378 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9483ae03-3fcd-41cc-8c50-be368fd9f1bd admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:38:26.953136 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-9483ae03-3fcd-41cc-8c50-be368fd9f1bd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2ea5d6767fe3420593c6d981055b069c'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:38:26.971995 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-9483ae03-3fcd-41cc-8c50-be368fd9f1bd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2ea5d6767fe3420593c6d981055b069c'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:38:26.973121 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-9483ae03-3fcd-41cc-8c50-be368fd9f1bd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2ea5d6767fe3420593c6d981055b069c'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:38:26.990264 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-9483ae03-3fcd-41cc-8c50-be368fd9f1bd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2ea5d6767fe3420593c6d981055b069c'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:38:26.991684 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-9483ae03-3fcd-41cc-8c50-be368fd9f1bd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2ea5d6767fe3420593c6d981055b069c'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:38:27.020190 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1049/2105] 10.222.0.33 () {58 vars in 1223 bytes} [Mon Jul 25 05:38:26 2022] DELETE /identity/v3/users/2ea5d6767fe3420593c6d981055b069c => generated 0 bytes in 233 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 05:38:27.024449 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1098dd6-66bc-48b2-88dd-c9494159f640 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:38:27.024988 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1098dd6-66bc-48b2-88dd-c9494159f640 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:38:27.025088 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1098dd6-66bc-48b2-88dd-c9494159f640 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:38:27.046378 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-d1098dd6-66bc-48b2-88dd-c9494159f640 None None] MFA Rules not processed for user `1eae540242d94c10b0ad33960a855f08`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:38:27.047613 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-d1098dd6-66bc-48b2-88dd-c9494159f640 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:38:27.133711 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1057/2106] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 25 05:38:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 05:38:27.169872 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9483ae03-3fcd-41cc-8c50-be368fd9f1bd admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:38:27.171414 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-9483ae03-3fcd-41cc-8c50-be368fd9f1bd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:38:27.216122 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:38:27.265779 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9483ae03-3fcd-41cc-8c50-be368fd9f1bd admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:38:27.269594 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-261ee809-02d6-4722-92a9-6990e758181e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b2931200f99e43d79a50a9f6e02738e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92afba266cce4d1bad4134c351e74be3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:38:27.270329 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261ee809-02d6-4722-92a9-6990e758181e service neutron] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:38:27.270714 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261ee809-02d6-4722-92a9-6990e758181e service neutron] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:38:27.271086 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261ee809-02d6-4722-92a9-6990e758181e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:38:27.283624 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-261ee809-02d6-4722-92a9-6990e758181e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:38:27.285220 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-261ee809-02d6-4722-92a9-6990e758181e service neutron] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:38:27.296436 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1050/2107] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jul 25 05:38:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 05:38:27.927380 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1098dd6-66bc-48b2-88dd-c9494159f640 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:38:27.929067 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-d1098dd6-66bc-48b2-88dd-c9494159f640 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:38:27.972333 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-261ee809-02d6-4722-92a9-6990e758181e service neutron] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:38:27.973125 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1098dd6-66bc-48b2-88dd-c9494159f640 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:38:27.973805 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-261ee809-02d6-4722-92a9-6990e758181e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:38:27.991174 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-168ead3a-fde5-4f99-84f4-b56df32619d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:38:27.992364 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168ead3a-fde5-4f99-84f4-b56df32619d3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:38:27.992658 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168ead3a-fde5-4f99-84f4-b56df32619d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:38:27.993673 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168ead3a-fde5-4f99-84f4-b56df32619d3 admin admin] PATH_INFO: `/v3/users/be5eb59db01b48448fef59467ee778ca` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:38:27.995424 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-168ead3a-fde5-4f99-84f4-b56df32619d3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=be5eb59db01b48448fef59467ee778ca)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:38:27.997077 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-261ee809-02d6-4722-92a9-6990e758181e service neutron] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:38:27.997636 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-168ead3a-fde5-4f99-84f4-b56df32619d3 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:38:28.011690 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff439f87-a674-4e47-ac04-f68311da955b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:38:28.014403 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff439f87-a674-4e47-ac04-f68311da955b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:38:28.015809 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff439f87-a674-4e47-ac04-f68311da955b admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:38:28.016351 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff439f87-a674-4e47-ac04-f68311da955b admin admin] PATH_INFO: `/v3/projects/04e3a31c2c3c488aadcce59baecefef2` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:38:28.019128 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff439f87-a674-4e47-ac04-f68311da955b admin admin] RBAC: Authorizing `identity:delete_project(project_id=04e3a31c2c3c488aadcce59baecefef2)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:38:28.021781 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff439f87-a674-4e47-ac04-f68311da955b admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:38:28.096159 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-ff439f87-a674-4e47-ac04-f68311da955b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '04e3a31c2c3c488aadcce59baecefef2', 'request_id': 'req-ff439f87-a674-4e47-ac04-f68311da955b'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:38:28.098097 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-168ead3a-fde5-4f99-84f4-b56df32619d3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'be5eb59db01b48448fef59467ee778ca'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:38:28.098952 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-168ead3a-fde5-4f99-84f4-b56df32619d3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'be5eb59db01b48448fef59467ee778ca'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:38:28.105071 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-ff439f87-a674-4e47-ac04-f68311da955b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '04e3a31c2c3c488aadcce59baecefef2', 'request_id': 'req-ff439f87-a674-4e47-ac04-f68311da955b'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:38:28.113015 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-168ead3a-fde5-4f99-84f4-b56df32619d3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'be5eb59db01b48448fef59467ee778ca'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:38:28.115990 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-ff439f87-a674-4e47-ac04-f68311da955b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '04e3a31c2c3c488aadcce59baecefef2', 'request_id': 'req-ff439f87-a674-4e47-ac04-f68311da955b'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:38:28.125563 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=91,keystone:INSERT=1,keystone:DELETE=8 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:38:28.133244 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-168ead3a-fde5-4f99-84f4-b56df32619d3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'be5eb59db01b48448fef59467ee778ca'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:38:28.134703 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-168ead3a-fde5-4f99-84f4-b56df32619d3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'be5eb59db01b48448fef59467ee778ca'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:38:28.137044 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-ff439f87-a674-4e47-ac04-f68311da955b admin admin] The token cache is being invalidate because project 04e3a31c2c3c488aadcce59baecefef2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80248) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 05:38:28.138938 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-ff439f87-a674-4e47-ac04-f68311da955b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:38:28.140059 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-ff439f87-a674-4e47-ac04-f68311da955b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:38:28.142035 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1051/2108] 10.222.0.33 () {58 vars in 1232 bytes} [Mon Jul 25 05:38:27 2022] DELETE /identity/v3/projects/04e3a31c2c3c488aadcce59baecefef2 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 05:38:28.147290 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb9d1a3-0157-4638-ba9d-6ad1dbb7e705 None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:38:28.147496 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb9d1a3-0157-4638-ba9d-6ad1dbb7e705 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:38:28.147866 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb9d1a3-0157-4638-ba9d-6ad1dbb7e705 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:38:28.171709 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-cfb9d1a3-0157-4638-ba9d-6ad1dbb7e705 None None] MFA Rules not processed for user `1eae540242d94c10b0ad33960a855f08`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:38:28.176872 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-cfb9d1a3-0157-4638-ba9d-6ad1dbb7e705 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:38:28.177031 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1058/2109] 10.222.0.33 () {58 vars in 1223 bytes} [Mon Jul 25 05:38:27 2022] DELETE /identity/v3/users/be5eb59db01b48448fef59467ee778ca => generated 0 bytes in 246 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 05:38:28.298541 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1052/2110] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 25 05:38:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 05:38:28.308042 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92710ca6-12d9-4fd8-bd13-53dacdbfc58f None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:38:28.308447 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92710ca6-12d9-4fd8-bd13-53dacdbfc58f None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:38:28.308867 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92710ca6-12d9-4fd8-bd13-53dacdbfc58f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:38:28.338307 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-92710ca6-12d9-4fd8-bd13-53dacdbfc58f None None] MFA Rules not processed for user `1eae540242d94c10b0ad33960a855f08`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:38:28.339674 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-92710ca6-12d9-4fd8-bd13-53dacdbfc58f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:38:28.352712 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1059/2111] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 25 05:38:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:38:28.353838 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb9d1a3-0157-4638-ba9d-6ad1dbb7e705 None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:38:28.372369 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb9d1a3-0157-4638-ba9d-6ad1dbb7e705 None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:38:28.379495 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51edf4e8-4fd8-44f4-b8c7-1209f699620b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:38:28.381341 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51edf4e8-4fd8-44f4-b8c7-1209f699620b admin admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:38:28.382026 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51edf4e8-4fd8-44f4-b8c7-1209f699620b admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:38:28.382712 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51edf4e8-4fd8-44f4-b8c7-1209f699620b admin admin] PATH_INFO: `/v3/domains` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:38:28.383640 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51edf4e8-4fd8-44f4-b8c7-1209f699620b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:38:28.384170 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51edf4e8-4fd8-44f4-b8c7-1209f699620b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:38:28.386522 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51edf4e8-4fd8-44f4-b8c7-1209f699620b admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:38:28.462768 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1053/2112] 10.222.0.33 () {58 vars in 1152 bytes} [Mon Jul 25 05:38:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 109 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 05:38:28.472914 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92710ca6-12d9-4fd8-bd13-53dacdbfc58f None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:38:28.481672 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92710ca6-12d9-4fd8-bd13-53dacdbfc58f None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:38:28.498957 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d98e47-5d68-4a76-98ea-3ff6b6dbabd3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:38:28.499725 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d98e47-5d68-4a76-98ea-3ff6b6dbabd3 admin admin] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:38:28.500079 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d98e47-5d68-4a76-98ea-3ff6b6dbabd3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:38:28.500429 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d98e47-5d68-4a76-98ea-3ff6b6dbabd3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:38:28.501306 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66d98e47-5d68-4a76-98ea-3ff6b6dbabd3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:38:28.509620 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66d98e47-5d68-4a76-98ea-3ff6b6dbabd3 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:38:28.600319 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1060/2113] 10.222.0.33 () {60 vars in 1152 bytes} [Mon Jul 25 05:38:28 2022] POST /identity/v3/projects => generated 385 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 05:38:28.604956 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51edf4e8-4fd8-44f4-b8c7-1209f699620b admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:38:28.619944 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51edf4e8-4fd8-44f4-b8c7-1209f699620b admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:38:28.658071 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d98e47-5d68-4a76-98ea-3ff6b6dbabd3 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:38:28.659960 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-66d98e47-5d68-4a76-98ea-3ff6b6dbabd3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:38:28.694255 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:38:28.741411 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed6baf9-73c9-4a47-80ef-7cece211825f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:38:28.742102 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed6baf9-73c9-4a47-80ef-7cece211825f admin admin] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:38:28.742386 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed6baf9-73c9-4a47-80ef-7cece211825f admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:38:28.742663 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed6baf9-73c9-4a47-80ef-7cece211825f admin admin] PATH_INFO: `/v3/users` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:38:28.743581 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ed6baf9-73c9-4a47-80ef-7cece211825f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:38:28.745649 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ed6baf9-73c9-4a47-80ef-7cece211825f admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:38:28.759141 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d98e47-5d68-4a76-98ea-3ff6b6dbabd3 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:38:28.765878 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f72a842-2d6c-490b-b7c9-11e7558d22d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:38:28.766880 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f72a842-2d6c-490b-b7c9-11e7558d22d5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:38:28.767488 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f72a842-2d6c-490b-b7c9-11e7558d22d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:38:28.768232 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f72a842-2d6c-490b-b7c9-11e7558d22d5 admin admin] PATH_INFO: `/v3/projects/b9b8418e171a4f92b5b99879dcfc6581` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:38:28.769560 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f72a842-2d6c-490b-b7c9-11e7558d22d5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b9b8418e171a4f92b5b99879dcfc6581)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:38:28.772317 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f72a842-2d6c-490b-b7c9-11e7558d22d5 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:38:28.783666 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1054/2114] 10.222.0.33 () {60 vars in 1143 bytes} [Mon Jul 25 05:38:28 2022] POST /identity/v3/users => generated 293 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 05:38:28.785744 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed6baf9-73c9-4a47-80ef-7cece211825f admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:38:28.797851 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed6baf9-73c9-4a47-80ef-7cece211825f admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:38:28.802687 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10086cc7-93b3-4031-9650-0e334ea2d2cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:38:28.803500 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10086cc7-93b3-4031-9650-0e334ea2d2cd admin admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:38:28.803876 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10086cc7-93b3-4031-9650-0e334ea2d2cd admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:38:28.804233 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10086cc7-93b3-4031-9650-0e334ea2d2cd admin admin] PATH_INFO: `/v3/roles` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:38:28.805089 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10086cc7-93b3-4031-9650-0e334ea2d2cd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:38:28.806779 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10086cc7-93b3-4031-9650-0e334ea2d2cd admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:38:28.818709 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1055/2115] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jul 25 05:38:28 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 05:38:28.823013 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10086cc7-93b3-4031-9650-0e334ea2d2cd admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:38:28.835635 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10086cc7-93b3-4031-9650-0e334ea2d2cd admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:38:28.839324 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-4f72a842-2d6c-490b-b7c9-11e7558d22d5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b9b8418e171a4f92b5b99879dcfc6581', 'request_id': 'req-4f72a842-2d6c-490b-b7c9-11e7558d22d5'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:38:28.846887 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-4f72a842-2d6c-490b-b7c9-11e7558d22d5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b9b8418e171a4f92b5b99879dcfc6581', 'request_id': 'req-4f72a842-2d6c-490b-b7c9-11e7558d22d5'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:38:28.853607 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-4f72a842-2d6c-490b-b7c9-11e7558d22d5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b9b8418e171a4f92b5b99879dcfc6581', 'request_id': 'req-4f72a842-2d6c-490b-b7c9-11e7558d22d5'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:38:28.871171 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-4f72a842-2d6c-490b-b7c9-11e7558d22d5 admin admin] The token cache is being invalidate because project b9b8418e171a4f92b5b99879dcfc6581 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80249) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 05:38:28.871457 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-4f72a842-2d6c-490b-b7c9-11e7558d22d5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:38:28.872767 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-4f72a842-2d6c-490b-b7c9-11e7558d22d5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:38:28.874515 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1061/2116] 10.222.0.33 () {58 vars in 1232 bytes} [Mon Jul 25 05:38:28 2022] DELETE /identity/v3/projects/b9b8418e171a4f92b5b99879dcfc6581 => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 25 05:38:28.878334 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f72a842-2d6c-490b-b7c9-11e7558d22d5 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:38:28.880068 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-4f72a842-2d6c-490b-b7c9-11e7558d22d5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:38:28.901788 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b54111c-b0f7-480e-b72e-136948475ff9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:38:28.902386 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b54111c-b0f7-480e-b72e-136948475ff9 admin admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:38:28.902677 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b54111c-b0f7-480e-b72e-136948475ff9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:38:28.903656 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b54111c-b0f7-480e-b72e-136948475ff9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:38:28.904821 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b54111c-b0f7-480e-b72e-136948475ff9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:38:28.906436 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b54111c-b0f7-480e-b72e-136948475ff9 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:38:28.922592 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1056/2117] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jul 25 05:38:28 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 25 05:38:28.923637 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b54111c-b0f7-480e-b72e-136948475ff9 admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:38:28.925506 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-5b54111c-b0f7-480e-b72e-136948475ff9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:38:28.935039 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=88,keystone:DELETE=9,keystone:INSERT=3 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:38:28.946691 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b54111c-b0f7-480e-b72e-136948475ff9 admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:38:28.952278 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df2914d-2c06-4e20-80f0-76295fe3cc08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:38:28.953323 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df2914d-2c06-4e20-80f0-76295fe3cc08 admin admin] REQUEST_METHOD: `PUT` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:38:28.953606 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df2914d-2c06-4e20-80f0-76295fe3cc08 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:38:28.953883 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df2914d-2c06-4e20-80f0-76295fe3cc08 admin admin] PATH_INFO: `/v3/projects/2c93153688f646fca7018d9edfe5574b/users/a537b33ba94140b6910a738a4028d2a1/roles/5074e74e35f54d4b9bb415c34999a82a` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:38:28.960101 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f72a842-2d6c-490b-b7c9-11e7558d22d5 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:38:28.965092 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5114a1a-db45-40f7-9aad-19f3be1bb3df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:38:28.966692 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5114a1a-db45-40f7-9aad-19f3be1bb3df admin admin] REQUEST_METHOD: `DELETE` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:38:28.967029 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5114a1a-db45-40f7-9aad-19f3be1bb3df admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:38:28.967379 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5114a1a-db45-40f7-9aad-19f3be1bb3df admin admin] PATH_INFO: `/v3/users/357148a4c5564dc3ab9b2273513ec470` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:38:28.969191 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5114a1a-db45-40f7-9aad-19f3be1bb3df admin admin] RBAC: Authorizing `identity:delete_user(user_id=357148a4c5564dc3ab9b2273513ec470)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:38:28.971481 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5114a1a-db45-40f7-9aad-19f3be1bb3df admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:38:28.982096 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6df2914d-2c06-4e20-80f0-76295fe3cc08 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2c93153688f646fca7018d9edfe5574b, user_id=a537b33ba94140b6910a738a4028d2a1, role_id=5074e74e35f54d4b9bb415c34999a82a)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:38:28.985658 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6df2914d-2c06-4e20-80f0-76295fe3cc08 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:38:29.008761 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1057/2118] 10.222.0.33 () {60 vars in 1482 bytes} [Mon Jul 25 05:38:28 2022] PUT /identity/v3/projects/2c93153688f646fca7018d9edfe5574b/users/a537b33ba94140b6910a738a4028d2a1/roles/5074e74e35f54d4b9bb415c34999a82a => generated 0 bytes in 85 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 05:38:29.019528 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc25cff5-0e42-48de-baa3-6a0930fb3472 None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:38:29.020903 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc25cff5-0e42-48de-baa3-6a0930fb3472 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:38:29.022567 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc25cff5-0e42-48de-baa3-6a0930fb3472 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:38:29.091707 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-d5114a1a-db45-40f7-9aad-19f3be1bb3df admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '357148a4c5564dc3ab9b2273513ec470'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:38:29.092952 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-d5114a1a-db45-40f7-9aad-19f3be1bb3df admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '357148a4c5564dc3ab9b2273513ec470'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:38:29.094461 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-cc25cff5-0e42-48de-baa3-6a0930fb3472 None None] MFA Rules not processed for user `a537b33ba94140b6910a738a4028d2a1`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:38:29.095299 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-cc25cff5-0e42-48de-baa3-6a0930fb3472 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:38:29.114919 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-d5114a1a-db45-40f7-9aad-19f3be1bb3df admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '357148a4c5564dc3ab9b2273513ec470'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:38:29.155290 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-d5114a1a-db45-40f7-9aad-19f3be1bb3df admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '357148a4c5564dc3ab9b2273513ec470'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:38:29.155974 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-d5114a1a-db45-40f7-9aad-19f3be1bb3df admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '357148a4c5564dc3ab9b2273513ec470'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:38:29.192861 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1062/2119] 10.222.0.33 () {58 vars in 1223 bytes} [Mon Jul 25 05:38:28 2022] DELETE /identity/v3/users/357148a4c5564dc3ab9b2273513ec470 => generated 0 bytes in 308 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 05:38:29.217439 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1058/2120] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 25 05:38:29 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 05:38:29.222383 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5114a1a-db45-40f7-9aad-19f3be1bb3df admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:38:29.225447 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-d5114a1a-db45-40f7-9aad-19f3be1bb3df admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:38:29.312258 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5114a1a-db45-40f7-9aad-19f3be1bb3df admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:38:29.318122 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960b3501-4c1b-4afd-aaa2-acf52bd72f04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:38:29.319121 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960b3501-4c1b-4afd-aaa2-acf52bd72f04 admin admin] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:38:29.319440 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960b3501-4c1b-4afd-aaa2-acf52bd72f04 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:38:29.319834 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960b3501-4c1b-4afd-aaa2-acf52bd72f04 admin admin] PATH_INFO: `/v3/projects` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:38:29.321019 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-960b3501-4c1b-4afd-aaa2-acf52bd72f04 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:38:29.322989 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-960b3501-4c1b-4afd-aaa2-acf52bd72f04 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:38:29.362049 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc25cff5-0e42-48de-baa3-6a0930fb3472 None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:38:29.362589 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1063/2121] 10.222.0.33 () {60 vars in 1152 bytes} [Mon Jul 25 05:38:29 2022] POST /identity/v3/projects => generated 383 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 05:38:29.379618 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:38:29.446821 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc25cff5-0e42-48de-baa3-6a0930fb3472 None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:38:29.523366 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960b3501-4c1b-4afd-aaa2-acf52bd72f04 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:38:29.525364 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-960b3501-4c1b-4afd-aaa2-acf52bd72f04 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:38:29.558163 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff0713c4-f1a3-4d5a-bd53-4b3c4b458973 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:38:29.559399 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0713c4-f1a3-4d5a-bd53-4b3c4b458973 admin admin] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:38:29.559983 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0713c4-f1a3-4d5a-bd53-4b3c4b458973 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:38:29.560941 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0713c4-f1a3-4d5a-bd53-4b3c4b458973 admin admin] PATH_INFO: `/v3/users` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:38:29.562869 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff0713c4-f1a3-4d5a-bd53-4b3c4b458973 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:38:29.568349 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff0713c4-f1a3-4d5a-bd53-4b3c4b458973 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:38:29.609406 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1059/2122] 10.222.0.33 () {60 vars in 1143 bytes} [Mon Jul 25 05:38:29 2022] POST /identity/v3/users => generated 292 bytes in 248 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 25 05:38:29.612689 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff0713c4-f1a3-4d5a-bd53-4b3c4b458973 admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:38:29.631513 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff0713c4-f1a3-4d5a-bd53-4b3c4b458973 admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:38:29.644376 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749e37d7-5514-4a28-a4b1-f625423ca1cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:38:29.645554 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749e37d7-5514-4a28-a4b1-f625423ca1cb admin admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:38:29.646035 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749e37d7-5514-4a28-a4b1-f625423ca1cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:38:29.646133 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749e37d7-5514-4a28-a4b1-f625423ca1cb admin admin] PATH_INFO: `/v3/roles` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:38:29.648552 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-749e37d7-5514-4a28-a4b1-f625423ca1cb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:38:29.650520 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-749e37d7-5514-4a28-a4b1-f625423ca1cb admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:38:29.656669 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:38:29.709659 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1060/2123] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jul 25 05:38:29 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 05:38:29.717826 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749e37d7-5514-4a28-a4b1-f625423ca1cb admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:38:29.728417 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960b3501-4c1b-4afd-aaa2-acf52bd72f04 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:38:29.737599 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea9b1b1-a2ba-4518-a6a4-e0e2a055fc1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:38:29.738652 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea9b1b1-a2ba-4518-a6a4-e0e2a055fc1e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:38:29.738941 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea9b1b1-a2ba-4518-a6a4-e0e2a055fc1e admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:38:29.739222 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea9b1b1-a2ba-4518-a6a4-e0e2a055fc1e admin admin] PATH_INFO: `/v3/projects/d9121e3b20714e6bb1930dd2d0d6be96` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:38:29.741976 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749e37d7-5514-4a28-a4b1-f625423ca1cb admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:38:29.743299 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ea9b1b1-a2ba-4518-a6a4-e0e2a055fc1e admin admin] RBAC: Authorizing `identity:delete_project(project_id=d9121e3b20714e6bb1930dd2d0d6be96)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:38:29.745952 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ea9b1b1-a2ba-4518-a6a4-e0e2a055fc1e admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:38:29.754920 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2069e1e8-3ce1-440f-9d1c-e8d8905356b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:38:29.755602 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2069e1e8-3ce1-440f-9d1c-e8d8905356b6 admin admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:38:29.755887 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2069e1e8-3ce1-440f-9d1c-e8d8905356b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:38:29.756165 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2069e1e8-3ce1-440f-9d1c-e8d8905356b6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:38:29.756957 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2069e1e8-3ce1-440f-9d1c-e8d8905356b6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:38:29.758560 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2069e1e8-3ce1-440f-9d1c-e8d8905356b6 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:38:29.776094 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1061/2124] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jul 25 05:38:29 2022] GET /identity/v3/roles => generated 1420 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 05:38:29.780639 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2069e1e8-3ce1-440f-9d1c-e8d8905356b6 admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:38:29.792277 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2069e1e8-3ce1-440f-9d1c-e8d8905356b6 admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:38:29.811147 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da1b0774-6629-479b-9c91-7538613498b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:38:29.812175 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1b0774-6629-479b-9c91-7538613498b9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:38:29.812536 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1b0774-6629-479b-9c91-7538613498b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:38:29.813085 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1b0774-6629-479b-9c91-7538613498b9 admin admin] PATH_INFO: `/v3/projects/6935284cd57940499a1e24401a9f2aa6/users/56965a3203ef4d3f8cff806207b1a806/roles/5074e74e35f54d4b9bb415c34999a82a` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:38:29.834181 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da1b0774-6629-479b-9c91-7538613498b9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6935284cd57940499a1e24401a9f2aa6, user_id=56965a3203ef4d3f8cff806207b1a806, role_id=5074e74e35f54d4b9bb415c34999a82a)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:38:29.837622 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da1b0774-6629-479b-9c91-7538613498b9 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:38:29.860040 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1062/2125] 10.222.0.33 () {60 vars in 1482 bytes} [Mon Jul 25 05:38:29 2022] PUT /identity/v3/projects/6935284cd57940499a1e24401a9f2aa6/users/56965a3203ef4d3f8cff806207b1a806/roles/5074e74e35f54d4b9bb415c34999a82a => generated 0 bytes in 80 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 25 05:38:29.870381 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f045135e-62d6-472f-ae58-295d76445e6d None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:38:29.870677 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f045135e-62d6-472f-ae58-295d76445e6d None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:38:29.870957 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f045135e-62d6-472f-ae58-295d76445e6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:38:29.875928 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-6ea9b1b1-a2ba-4518-a6a4-e0e2a055fc1e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd9121e3b20714e6bb1930dd2d0d6be96', 'request_id': 'req-6ea9b1b1-a2ba-4518-a6a4-e0e2a055fc1e'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:38:29.883924 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-6ea9b1b1-a2ba-4518-a6a4-e0e2a055fc1e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd9121e3b20714e6bb1930dd2d0d6be96', 'request_id': 'req-6ea9b1b1-a2ba-4518-a6a4-e0e2a055fc1e'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:38:29.898339 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-6ea9b1b1-a2ba-4518-a6a4-e0e2a055fc1e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd9121e3b20714e6bb1930dd2d0d6be96', 'request_id': 'req-6ea9b1b1-a2ba-4518-a6a4-e0e2a055fc1e'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:38:29.918765 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-f045135e-62d6-472f-ae58-295d76445e6d None None] MFA Rules not processed for user `56965a3203ef4d3f8cff806207b1a806`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:38:29.918959 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-6ea9b1b1-a2ba-4518-a6a4-e0e2a055fc1e admin admin] The token cache is being invalidate because project d9121e3b20714e6bb1930dd2d0d6be96 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80249) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 05:38:29.920123 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-6ea9b1b1-a2ba-4518-a6a4-e0e2a055fc1e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:38:29.920236 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-f045135e-62d6-472f-ae58-295d76445e6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:38:29.920911 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-6ea9b1b1-a2ba-4518-a6a4-e0e2a055fc1e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:38:29.927333 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1064/2126] 10.222.0.33 () {58 vars in 1232 bytes} [Mon Jul 25 05:38:29 2022] DELETE /identity/v3/projects/d9121e3b20714e6bb1930dd2d0d6be96 => generated 0 bytes in 401 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 05:38:29.937711 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c956e9a1-a2cd-4d88-8c75-9931f5908134 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:38:29.938733 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c956e9a1-a2cd-4d88-8c75-9931f5908134 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:38:29.939037 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c956e9a1-a2cd-4d88-8c75-9931f5908134 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:38:29.962686 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-c956e9a1-a2cd-4d88-8c75-9931f5908134 None None] MFA Rules not processed for user `1eae540242d94c10b0ad33960a855f08`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:38:29.963963 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-c956e9a1-a2cd-4d88-8c75-9931f5908134 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:38:30.001021 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:38:30.016857 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1063/2127] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 25 05:38:29 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:38:30.020953 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f045135e-62d6-472f-ae58-295d76445e6d None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:38:30.022332 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-f045135e-62d6-472f-ae58-295d76445e6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:38:30.037448 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1065/2128] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 25 05:38:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:38:30.045221 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88add39-2882-413f-9b6a-93e3242df944 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:38:30.045518 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88add39-2882-413f-9b6a-93e3242df944 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:38:30.045808 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88add39-2882-413f-9b6a-93e3242df944 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:38:30.069623 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-d88add39-2882-413f-9b6a-93e3242df944 None None] MFA Rules not processed for user `1eae540242d94c10b0ad33960a855f08`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:38:30.071514 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-d88add39-2882-413f-9b6a-93e3242df944 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:38:30.085580 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1066/2129] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 25 05:38:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:38:30.087814 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d88add39-2882-413f-9b6a-93e3242df944 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:38:30.098643 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d88add39-2882-413f-9b6a-93e3242df944 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:38:30.105765 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e286ae-577e-4c23-a155-4735ae6a0379 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:38:30.107224 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e286ae-577e-4c23-a155-4735ae6a0379 admin admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:38:30.107670 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e286ae-577e-4c23-a155-4735ae6a0379 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:38:30.108088 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e286ae-577e-4c23-a155-4735ae6a0379 admin admin] PATH_INFO: `/v3/domains` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:38:30.109162 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8e286ae-577e-4c23-a155-4735ae6a0379 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:38:30.109573 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8e286ae-577e-4c23-a155-4735ae6a0379 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:38:30.111315 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8e286ae-577e-4c23-a155-4735ae6a0379 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:38:30.123835 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f045135e-62d6-472f-ae58-295d76445e6d None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:38:30.129171 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be79856-bf47-4b5d-ad65-4b56e7bb19e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:38:30.129868 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be79856-bf47-4b5d-ad65-4b56e7bb19e8 admin admin] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:38:30.130158 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be79856-bf47-4b5d-ad65-4b56e7bb19e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:38:30.130464 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be79856-bf47-4b5d-ad65-4b56e7bb19e8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:38:30.131197 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4be79856-bf47-4b5d-ad65-4b56e7bb19e8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:38:30.133850 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4be79856-bf47-4b5d-ad65-4b56e7bb19e8 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:38:30.140836 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1067/2130] 10.222.0.33 () {58 vars in 1152 bytes} [Mon Jul 25 05:38:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:38:30.144953 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e286ae-577e-4c23-a155-4735ae6a0379 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:38:30.158033 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e286ae-577e-4c23-a155-4735ae6a0379 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:38:30.164320 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76510336-6342-4036-b555-ff077210ca23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:38:30.165334 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76510336-6342-4036-b555-ff077210ca23 admin admin] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:38:30.167181 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76510336-6342-4036-b555-ff077210ca23 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:38:30.167657 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76510336-6342-4036-b555-ff077210ca23 admin admin] PATH_INFO: `/v3/projects` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:38:30.169132 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76510336-6342-4036-b555-ff077210ca23 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:38:30.171973 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76510336-6342-4036-b555-ff077210ca23 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:38:30.183826 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1064/2131] 10.222.0.33 () {60 vars in 1152 bytes} [Mon Jul 25 05:38:30 2022] POST /identity/v3/projects => generated 383 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 25 05:38:30.185591 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be79856-bf47-4b5d-ad65-4b56e7bb19e8 admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:38:30.222197 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1068/2132] 10.222.0.33 () {60 vars in 1152 bytes} [Mon Jul 25 05:38:30 2022] POST /identity/v3/projects => generated 377 bytes in 78 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 25 05:38:30.223132 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76510336-6342-4036-b555-ff077210ca23 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:38:30.225211 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-76510336-6342-4036-b555-ff077210ca23 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:38:30.309205 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:38:30.309982 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be79856-bf47-4b5d-ad65-4b56e7bb19e8 admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:38:30.347192 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76510336-6342-4036-b555-ff077210ca23 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:38:30.353526 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-befd3ae1-bb9f-41dd-853a-2acde7f732f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:38:30.354797 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-befd3ae1-bb9f-41dd-853a-2acde7f732f6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:38:30.355219 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-befd3ae1-bb9f-41dd-853a-2acde7f732f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:38:30.355598 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-befd3ae1-bb9f-41dd-853a-2acde7f732f6 admin admin] PATH_INFO: `/v3/users/ae21aa22a8e04173a9a5e7500a390be8` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:38:30.357925 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-befd3ae1-bb9f-41dd-853a-2acde7f732f6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ae21aa22a8e04173a9a5e7500a390be8)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:38:30.360386 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-befd3ae1-bb9f-41dd-853a-2acde7f732f6 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:38:30.383606 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e92900-8768-45fb-aabd-d0bd03be2a81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:38:30.384878 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e92900-8768-45fb-aabd-d0bd03be2a81 admin admin] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:38:30.385650 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e92900-8768-45fb-aabd-d0bd03be2a81 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:38:30.387666 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e92900-8768-45fb-aabd-d0bd03be2a81 admin admin] PATH_INFO: `/v3/users` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:38:30.388624 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4e92900-8768-45fb-aabd-d0bd03be2a81 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:38:30.390890 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4e92900-8768-45fb-aabd-d0bd03be2a81 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:38:30.422196 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-befd3ae1-bb9f-41dd-853a-2acde7f732f6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ae21aa22a8e04173a9a5e7500a390be8'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:38:30.422939 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-befd3ae1-bb9f-41dd-853a-2acde7f732f6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ae21aa22a8e04173a9a5e7500a390be8'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:38:30.423759 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1065/2133] 10.222.0.33 () {60 vars in 1143 bytes} [Mon Jul 25 05:38:30 2022] POST /identity/v3/users => generated 298 bytes in 239 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 25 05:38:30.429570 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e92900-8768-45fb-aabd-d0bd03be2a81 admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:38:30.433234 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-befd3ae1-bb9f-41dd-853a-2acde7f732f6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ae21aa22a8e04173a9a5e7500a390be8'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:38:30.441764 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e92900-8768-45fb-aabd-d0bd03be2a81 admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:38:30.452812 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-befd3ae1-bb9f-41dd-853a-2acde7f732f6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ae21aa22a8e04173a9a5e7500a390be8'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:38:30.453076 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aee7dd6-20d9-4ee9-a9d2-9962e1485670 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:38:30.453760 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-befd3ae1-bb9f-41dd-853a-2acde7f732f6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ae21aa22a8e04173a9a5e7500a390be8'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:38:30.453883 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aee7dd6-20d9-4ee9-a9d2-9962e1485670 admin admin] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:38:30.454138 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aee7dd6-20d9-4ee9-a9d2-9962e1485670 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:38:30.454419 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aee7dd6-20d9-4ee9-a9d2-9962e1485670 admin admin] PATH_INFO: `/v3/users` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:38:30.455763 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aee7dd6-20d9-4ee9-a9d2-9962e1485670 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:38:30.458122 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aee7dd6-20d9-4ee9-a9d2-9962e1485670 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:38:30.475655 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1069/2134] 10.222.0.33 () {58 vars in 1223 bytes} [Mon Jul 25 05:38:30 2022] DELETE /identity/v3/users/ae21aa22a8e04173a9a5e7500a390be8 => generated 0 bytes in 253 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 05:38:30.476595 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-befd3ae1-bb9f-41dd-853a-2acde7f732f6 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:38:30.481982 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:38:30.482128 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-befd3ae1-bb9f-41dd-853a-2acde7f732f6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:38:30.489063 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1066/2135] 10.222.0.33 () {60 vars in 1143 bytes} [Mon Jul 25 05:38:30 2022] POST /identity/v3/users => generated 289 bytes in 65 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 05:38:30.490712 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0195e3b6-09cd-4bb3-afb3-bf9c8e721458 None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:38:30.491424 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0195e3b6-09cd-4bb3-afb3-bf9c8e721458 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:38:30.491811 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0195e3b6-09cd-4bb3-afb3-bf9c8e721458 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:38:30.521319 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-0195e3b6-09cd-4bb3-afb3-bf9c8e721458 None None] MFA Rules not processed for user `1eae540242d94c10b0ad33960a855f08`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:38:30.522620 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-0195e3b6-09cd-4bb3-afb3-bf9c8e721458 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:38:30.599375 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-befd3ae1-bb9f-41dd-853a-2acde7f732f6 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:38:30.604421 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b20191a-7f8b-4bbf-96f0-26df7277eee8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:38:30.608289 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b20191a-7f8b-4bbf-96f0-26df7277eee8 admin admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:38:30.608981 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b20191a-7f8b-4bbf-96f0-26df7277eee8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:38:30.609537 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b20191a-7f8b-4bbf-96f0-26df7277eee8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:38:30.610574 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b20191a-7f8b-4bbf-96f0-26df7277eee8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:38:30.614693 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1067/2136] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 25 05:38:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:38:30.614895 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b20191a-7f8b-4bbf-96f0-26df7277eee8 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:38:30.621051 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0195e3b6-09cd-4bb3-afb3-bf9c8e721458 None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:38:30.624542 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-0195e3b6-09cd-4bb3-afb3-bf9c8e721458 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:38:30.635972 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1070/2137] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jul 25 05:38:30 2022] GET /identity/v3/roles => generated 1420 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 05:38:30.637102 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b20191a-7f8b-4bbf-96f0-26df7277eee8 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:38:30.639221 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-0b20191a-7f8b-4bbf-96f0-26df7277eee8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:38:30.651368 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0195e3b6-09cd-4bb3-afb3-bf9c8e721458 None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:38:30.662785 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b0f219-f3c5-499c-bdd3-b3355568d71d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:38:30.663998 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b0f219-f3c5-499c-bdd3-b3355568d71d admin admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:38:30.664724 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b0f219-f3c5-499c-bdd3-b3355568d71d admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:38:30.665053 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b0f219-f3c5-499c-bdd3-b3355568d71d admin admin] PATH_INFO: `/v3/roles` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:38:30.665930 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87b0f219-f3c5-499c-bdd3-b3355568d71d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:38:30.667866 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87b0f219-f3c5-499c-bdd3-b3355568d71d admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:38:30.688496 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1068/2138] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jul 25 05:38:30 2022] GET /identity/v3/roles => generated 1420 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 05:38:30.689439 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b0f219-f3c5-499c-bdd3-b3355568d71d admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:38:30.701081 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b0f219-f3c5-499c-bdd3-b3355568d71d admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:38:30.706789 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d36a7e-8667-4405-a8ba-3183f23bdf16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:38:30.707749 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d36a7e-8667-4405-a8ba-3183f23bdf16 admin admin] REQUEST_METHOD: `PUT` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:38:30.708037 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d36a7e-8667-4405-a8ba-3183f23bdf16 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:38:30.708311 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d36a7e-8667-4405-a8ba-3183f23bdf16 admin admin] PATH_INFO: `/v3/projects/ace7b756233b490a8f52550824f77d30/users/b67b27be926e42f3a076518a19a5e3c2/roles/39e5c666285a42fe8fc2c3afca6260d1` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:38:30.733016 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32d36a7e-8667-4405-a8ba-3183f23bdf16 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ace7b756233b490a8f52550824f77d30, user_id=b67b27be926e42f3a076518a19a5e3c2, role_id=39e5c666285a42fe8fc2c3afca6260d1)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:38:30.739081 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32d36a7e-8667-4405-a8ba-3183f23bdf16 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:38:30.764636 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1069/2139] 10.222.0.33 () {60 vars in 1482 bytes} [Mon Jul 25 05:38:30 2022] PUT /identity/v3/projects/ace7b756233b490a8f52550824f77d30/users/b67b27be926e42f3a076518a19a5e3c2/roles/39e5c666285a42fe8fc2c3afca6260d1 => generated 0 bytes in 72 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 05:38:30.765807 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d36a7e-8667-4405-a8ba-3183f23bdf16 admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:38:30.789697 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b20191a-7f8b-4bbf-96f0-26df7277eee8 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:38:30.817383 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:38:30.879430 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b437b6f-c2ac-4c07-8043-77ebf4fbb86e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b2931200f99e43d79a50a9f6e02738e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92afba266cce4d1bad4134c351e74be3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:38:30.888490 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d36a7e-8667-4405-a8ba-3183f23bdf16 admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:38:30.889062 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b437b6f-c2ac-4c07-8043-77ebf4fbb86e service neutron] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:38:30.889381 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b437b6f-c2ac-4c07-8043-77ebf4fbb86e service neutron] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:38:30.889667 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b437b6f-c2ac-4c07-8043-77ebf4fbb86e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:38:30.898047 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f695e2c2-54b2-456f-ac5a-b77803c9ed47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:38:30.902445 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f695e2c2-54b2-456f-ac5a-b77803c9ed47 admin admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:38:30.902799 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f695e2c2-54b2-456f-ac5a-b77803c9ed47 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:38:30.903567 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b437b6f-c2ac-4c07-8043-77ebf4fbb86e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:38:30.903928 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f695e2c2-54b2-456f-ac5a-b77803c9ed47 admin admin] PATH_INFO: `/v3/roles` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:38:30.904700 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f695e2c2-54b2-456f-ac5a-b77803c9ed47 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:38:30.905759 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b437b6f-c2ac-4c07-8043-77ebf4fbb86e service neutron] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:38:30.906363 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f695e2c2-54b2-456f-ac5a-b77803c9ed47 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:38:30.917445 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1071/2140] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jul 25 05:38:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 281 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 05:38:30.918411 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b437b6f-c2ac-4c07-8043-77ebf4fbb86e service neutron] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:38:30.923578 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1070/2141] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jul 25 05:38:30 2022] GET /identity/v3/roles => generated 1420 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 05:38:30.927733 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f695e2c2-54b2-456f-ac5a-b77803c9ed47 admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:38:30.938127 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b437b6f-c2ac-4c07-8043-77ebf4fbb86e service neutron] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:38:30.945431 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f695e2c2-54b2-456f-ac5a-b77803c9ed47 admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:38:30.952868 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcab69c9-4359-4bdf-8eb3-4d0e4211260c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:38:30.954727 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcab69c9-4359-4bdf-8eb3-4d0e4211260c admin admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:38:30.955616 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcab69c9-4359-4bdf-8eb3-4d0e4211260c admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:38:30.956741 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcab69c9-4359-4bdf-8eb3-4d0e4211260c admin admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:38:30.957611 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcab69c9-4359-4bdf-8eb3-4d0e4211260c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:38:30.960439 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcab69c9-4359-4bdf-8eb3-4d0e4211260c admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:38:30.963797 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f636fa41-911a-4df5-873f-26f387839658 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:38:30.964878 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f636fa41-911a-4df5-873f-26f387839658 admin admin] REQUEST_METHOD: `PUT` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:38:30.965268 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f636fa41-911a-4df5-873f-26f387839658 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:38:30.966757 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f636fa41-911a-4df5-873f-26f387839658 admin admin] PATH_INFO: `/v3/projects/35d2f0aee1784bdba7727f70fd41c28e/users/30e5711bc03a4ca888084a599f217a0f/roles/5074e74e35f54d4b9bb415c34999a82a` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:38:31.013140 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1072/2142] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jul 25 05:38:30 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 05:38:31.017762 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcab69c9-4359-4bdf-8eb3-4d0e4211260c admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:38:31.035159 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f636fa41-911a-4df5-873f-26f387839658 admin admin] RBAC: Authorizing `identity:create_grant(project_id=35d2f0aee1784bdba7727f70fd41c28e, user_id=30e5711bc03a4ca888084a599f217a0f, role_id=5074e74e35f54d4b9bb415c34999a82a)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:38:31.038038 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f636fa41-911a-4df5-873f-26f387839658 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:38:31.044589 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcab69c9-4359-4bdf-8eb3-4d0e4211260c admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:38:31.057282 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c630aa-24ca-4609-ad15-ede90bf94f15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:38:31.060135 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c630aa-24ca-4609-ad15-ede90bf94f15 admin admin] REQUEST_METHOD: `PUT` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:38:31.060561 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c630aa-24ca-4609-ad15-ede90bf94f15 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:38:31.060884 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c630aa-24ca-4609-ad15-ede90bf94f15 admin admin] PATH_INFO: `/v3/projects/ace7b756233b490a8f52550824f77d30/users/b67b27be926e42f3a076518a19a5e3c2/roles/39e5c666285a42fe8fc2c3afca6260d1` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:38:31.063676 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1071/2143] 10.222.0.33 () {60 vars in 1482 bytes} [Mon Jul 25 05:38:30 2022] PUT /identity/v3/projects/35d2f0aee1784bdba7727f70fd41c28e/users/30e5711bc03a4ca888084a599f217a0f/roles/5074e74e35f54d4b9bb415c34999a82a => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 25 05:38:31.066627 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08c630aa-24ca-4609-ad15-ede90bf94f15 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ace7b756233b490a8f52550824f77d30, user_id=b67b27be926e42f3a076518a19a5e3c2, role_id=39e5c666285a42fe8fc2c3afca6260d1)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:38:31.073681 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08c630aa-24ca-4609-ad15-ede90bf94f15 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:38:31.077362 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637f845d-1bcd-4411-83af-afffa41b943d None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:38:31.077804 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637f845d-1bcd-4411-83af-afffa41b943d None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:38:31.078185 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637f845d-1bcd-4411-83af-afffa41b943d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:38:31.116099 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1073/2144] 10.222.0.33 () {60 vars in 1482 bytes} [Mon Jul 25 05:38:31 2022] PUT /identity/v3/projects/ace7b756233b490a8f52550824f77d30/users/b67b27be926e42f3a076518a19a5e3c2/roles/39e5c666285a42fe8fc2c3afca6260d1 => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 05:38:31.182429 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-637f845d-1bcd-4411-83af-afffa41b943d None None] MFA Rules not processed for user `30e5711bc03a4ca888084a599f217a0f`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:38:31.183781 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-637f845d-1bcd-4411-83af-afffa41b943d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:38:31.223487 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4e2ab6-5c93-45bb-8c3d-15ecbbc4e156 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:38:31.223487 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4e2ab6-5c93-45bb-8c3d-15ecbbc4e156 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:38:31.223487 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4e2ab6-5c93-45bb-8c3d-15ecbbc4e156 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:38:31.263694 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:38:31.282490 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-1d4e2ab6-5c93-45bb-8c3d-15ecbbc4e156 None None] MFA Rules not processed for user `b67b27be926e42f3a076518a19a5e3c2`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:38:31.283734 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-1d4e2ab6-5c93-45bb-8c3d-15ecbbc4e156 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:38:31.394007 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1072/2145] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 25 05:38:31 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 314 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:38:31.404388 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-637f845d-1bcd-4411-83af-afffa41b943d None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:38:31.406401 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-637f845d-1bcd-4411-83af-afffa41b943d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:38:31.471865 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1074/2146] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 25 05:38:31 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 251 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 05:38:31.525732 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d4e2ab6-5c93-45bb-8c3d-15ecbbc4e156 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:38:31.527335 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-1d4e2ab6-5c93-45bb-8c3d-15ecbbc4e156 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:38:31.542843 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-637f845d-1bcd-4411-83af-afffa41b943d None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:38:31.555382 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83dd9057-127e-4ad9-a4d0-6f185b5ffcfd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db57b478c25643dbaf936f90f6e17d0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92afba266cce4d1bad4134c351e74be3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:38:31.564160 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83dd9057-127e-4ad9-a4d0-6f185b5ffcfd service cinder] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:38:31.564160 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83dd9057-127e-4ad9-a4d0-6f185b5ffcfd service cinder] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:38:31.564160 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83dd9057-127e-4ad9-a4d0-6f185b5ffcfd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:38:31.576501 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83dd9057-127e-4ad9-a4d0-6f185b5ffcfd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:38:31.578269 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83dd9057-127e-4ad9-a4d0-6f185b5ffcfd service cinder] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:38:31.592221 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1073/2147] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jul 25 05:38:31 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 186 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 05:38:31.601720 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83dd9057-127e-4ad9-a4d0-6f185b5ffcfd service cinder] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:38:31.607687 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-83dd9057-127e-4ad9-a4d0-6f185b5ffcfd service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:38:31.633374 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83dd9057-127e-4ad9-a4d0-6f185b5ffcfd service cinder] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:38:31.644740 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edca8bda-765e-4a4d-8af9-92baa692009b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db57b478c25643dbaf936f90f6e17d0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92afba266cce4d1bad4134c351e74be3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:38:31.646420 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edca8bda-765e-4a4d-8af9-92baa692009b service cinder] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:38:31.646803 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edca8bda-765e-4a4d-8af9-92baa692009b service cinder] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:38:31.647160 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edca8bda-765e-4a4d-8af9-92baa692009b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:38:31.650102 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d4e2ab6-5c93-45bb-8c3d-15ecbbc4e156 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:38:31.653143 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-edca8bda-765e-4a4d-8af9-92baa692009b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:38:31.662301 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-664d01e8-b251-4dad-8244-7077481c75ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:38:31.665695 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664d01e8-b251-4dad-8244-7077481c75ba admin admin] REQUEST_METHOD: `DELETE` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:38:31.665987 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664d01e8-b251-4dad-8244-7077481c75ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:38:31.666450 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664d01e8-b251-4dad-8244-7077481c75ba admin admin] PATH_INFO: `/v3/users/bf61ffac0e3143f6b44971158bd7a553` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:38:31.670557 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-664d01e8-b251-4dad-8244-7077481c75ba admin admin] RBAC: Authorizing `identity:delete_user(user_id=bf61ffac0e3143f6b44971158bd7a553)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:38:31.674312 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-664d01e8-b251-4dad-8244-7077481c75ba admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:38:31.707826 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=98,keystone:INSERT=1,keystone:DELETE=1 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:38:31.765283 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-664d01e8-b251-4dad-8244-7077481c75ba admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bf61ffac0e3143f6b44971158bd7a553'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:38:31.767324 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-664d01e8-b251-4dad-8244-7077481c75ba admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bf61ffac0e3143f6b44971158bd7a553'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:38:31.783274 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edca8bda-765e-4a4d-8af9-92baa692009b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:38:31.785859 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edca8bda-765e-4a4d-8af9-92baa692009b service cinder] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:38:31.790598 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-664d01e8-b251-4dad-8244-7077481c75ba admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bf61ffac0e3143f6b44971158bd7a553'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:38:31.801506 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1074/2148] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jul 25 05:38:31 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 208 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 05:38:31.815038 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-664d01e8-b251-4dad-8244-7077481c75ba admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bf61ffac0e3143f6b44971158bd7a553'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:38:31.816170 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-664d01e8-b251-4dad-8244-7077481c75ba admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bf61ffac0e3143f6b44971158bd7a553'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:38:31.821692 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edca8bda-765e-4a4d-8af9-92baa692009b service cinder] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:38:31.823766 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-edca8bda-765e-4a4d-8af9-92baa692009b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:38:31.844925 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1075/2149] 10.222.0.33 () {58 vars in 1223 bytes} [Mon Jul 25 05:38:31 2022] DELETE /identity/v3/users/bf61ffac0e3143f6b44971158bd7a553 => generated 0 bytes in 319 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 25 05:38:31.858080 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edca8bda-765e-4a4d-8af9-92baa692009b service cinder] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:38:31.959797 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf5178a-eb46-4df9-bb8b-52a3f7ad69b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:38:31.961310 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf5178a-eb46-4df9-bb8b-52a3f7ad69b0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:38:31.961701 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf5178a-eb46-4df9-bb8b-52a3f7ad69b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:38:31.962172 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf5178a-eb46-4df9-bb8b-52a3f7ad69b0 admin admin] PATH_INFO: `/v3/projects/ae670cedc6c149a7bedea817cb280406` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:38:31.963693 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdf5178a-eb46-4df9-bb8b-52a3f7ad69b0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ae670cedc6c149a7bedea817cb280406)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:38:31.967035 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdf5178a-eb46-4df9-bb8b-52a3f7ad69b0 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:38:31.980193 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:38:32.068485 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-cdf5178a-eb46-4df9-bb8b-52a3f7ad69b0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ae670cedc6c149a7bedea817cb280406', 'request_id': 'req-cdf5178a-eb46-4df9-bb8b-52a3f7ad69b0'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:38:32.084719 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-cdf5178a-eb46-4df9-bb8b-52a3f7ad69b0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ae670cedc6c149a7bedea817cb280406', 'request_id': 'req-cdf5178a-eb46-4df9-bb8b-52a3f7ad69b0'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:38:32.094788 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-cdf5178a-eb46-4df9-bb8b-52a3f7ad69b0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ae670cedc6c149a7bedea817cb280406', 'request_id': 'req-cdf5178a-eb46-4df9-bb8b-52a3f7ad69b0'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:38:32.111327 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-664d01e8-b251-4dad-8244-7077481c75ba admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:38:32.112848 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-664d01e8-b251-4dad-8244-7077481c75ba admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:38:32.113698 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-cdf5178a-eb46-4df9-bb8b-52a3f7ad69b0 admin admin] The token cache is being invalidate because project ae670cedc6c149a7bedea817cb280406 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80248) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 05:38:32.114040 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-cdf5178a-eb46-4df9-bb8b-52a3f7ad69b0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:38:32.114682 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-cdf5178a-eb46-4df9-bb8b-52a3f7ad69b0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:38:32.116874 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1075/2150] 10.222.0.33 () {58 vars in 1232 bytes} [Mon Jul 25 05:38:31 2022] DELETE /identity/v3/projects/ae670cedc6c149a7bedea817cb280406 => generated 0 bytes in 295 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 05:38:32.126193 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf5178a-eb46-4df9-bb8b-52a3f7ad69b0 admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:38:32.131757 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-cdf5178a-eb46-4df9-bb8b-52a3f7ad69b0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:38:32.147911 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-664d01e8-b251-4dad-8244-7077481c75ba admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:38:32.228973 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc766764-d2c4-43d1-b474-17d557bda0ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:38:32.231979 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc766764-d2c4-43d1-b474-17d557bda0ef admin admin] REQUEST_METHOD: `DELETE` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:38:32.232344 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc766764-d2c4-43d1-b474-17d557bda0ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:38:32.232696 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc766764-d2c4-43d1-b474-17d557bda0ef admin admin] PATH_INFO: `/v3/projects/0dc32f383ef54d94aeb44d0bc3885614` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:38:32.234218 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf5178a-eb46-4df9-bb8b-52a3f7ad69b0 admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:38:32.234784 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc766764-d2c4-43d1-b474-17d557bda0ef admin admin] RBAC: Authorizing `identity:delete_project(project_id=0dc32f383ef54d94aeb44d0bc3885614)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:38:32.238174 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc766764-d2c4-43d1-b474-17d557bda0ef admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:38:32.243121 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15d979b-c745-4ceb-afb8-a5e658dbcb9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:38:32.244745 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15d979b-c745-4ceb-afb8-a5e658dbcb9d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:38:32.245073 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15d979b-c745-4ceb-afb8-a5e658dbcb9d admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:38:32.245351 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15d979b-c745-4ceb-afb8-a5e658dbcb9d admin admin] PATH_INFO: `/v3/users/5c7f3ad50365464e8eb77588525ee531` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:38:32.246675 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d15d979b-c745-4ceb-afb8-a5e658dbcb9d admin admin] RBAC: Authorizing `identity:delete_user(user_id=5c7f3ad50365464e8eb77588525ee531)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:38:32.248796 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d15d979b-c745-4ceb-afb8-a5e658dbcb9d admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:38:32.296768 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-dc766764-d2c4-43d1-b474-17d557bda0ef admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0dc32f383ef54d94aeb44d0bc3885614', 'request_id': 'req-dc766764-d2c4-43d1-b474-17d557bda0ef'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:38:32.306482 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-dc766764-d2c4-43d1-b474-17d557bda0ef admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0dc32f383ef54d94aeb44d0bc3885614', 'request_id': 'req-dc766764-d2c4-43d1-b474-17d557bda0ef'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:38:32.316984 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-dc766764-d2c4-43d1-b474-17d557bda0ef admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0dc32f383ef54d94aeb44d0bc3885614', 'request_id': 'req-dc766764-d2c4-43d1-b474-17d557bda0ef'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:38:32.329666 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-dc766764-d2c4-43d1-b474-17d557bda0ef admin admin] The token cache is being invalidate because project 0dc32f383ef54d94aeb44d0bc3885614 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80249) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 05:38:32.330043 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-dc766764-d2c4-43d1-b474-17d557bda0ef admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:38:32.330144 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-d15d979b-c745-4ceb-afb8-a5e658dbcb9d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5c7f3ad50365464e8eb77588525ee531'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:38:32.330762 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-dc766764-d2c4-43d1-b474-17d557bda0ef admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:38:32.333458 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1076/2151] 10.222.0.33 () {58 vars in 1232 bytes} [Mon Jul 25 05:38:32 2022] DELETE /identity/v3/projects/0dc32f383ef54d94aeb44d0bc3885614 => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 25 05:38:32.338190 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37fb9300-769b-4a41-a649-363b1d101586 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:38:32.338491 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37fb9300-769b-4a41-a649-363b1d101586 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:38:32.338777 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37fb9300-769b-4a41-a649-363b1d101586 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:38:32.346221 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-d15d979b-c745-4ceb-afb8-a5e658dbcb9d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5c7f3ad50365464e8eb77588525ee531'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:38:32.346996 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-d15d979b-c745-4ceb-afb8-a5e658dbcb9d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5c7f3ad50365464e8eb77588525ee531'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:38:32.355998 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-d15d979b-c745-4ceb-afb8-a5e658dbcb9d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5c7f3ad50365464e8eb77588525ee531'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:38:32.356957 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-d15d979b-c745-4ceb-afb8-a5e658dbcb9d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5c7f3ad50365464e8eb77588525ee531'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:38:32.362586 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-37fb9300-769b-4a41-a649-363b1d101586 None None] MFA Rules not processed for user `1eae540242d94c10b0ad33960a855f08`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:38:32.363994 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-37fb9300-769b-4a41-a649-363b1d101586 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:38:32.370388 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1076/2152] 10.222.0.33 () {58 vars in 1223 bytes} [Mon Jul 25 05:38:32 2022] DELETE /identity/v3/users/5c7f3ad50365464e8eb77588525ee531 => generated 0 bytes in 247 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 05:38:32.388220 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:DELETE=16,keystone:SELECT=82,keystone:INSERT=2 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:38:32.448569 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1077/2153] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 25 05:38:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 05:38:32.458167 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dbf11bc-4118-421a-9b4f-3d668bbec188 None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:38:32.458466 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dbf11bc-4118-421a-9b4f-3d668bbec188 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:38:32.459855 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dbf11bc-4118-421a-9b4f-3d668bbec188 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:38:32.495009 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-8dbf11bc-4118-421a-9b4f-3d668bbec188 None None] MFA Rules not processed for user `1eae540242d94c10b0ad33960a855f08`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:38:32.496495 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-8dbf11bc-4118-421a-9b4f-3d668bbec188 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:38:32.508655 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:38:32.582023 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1077/2154] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 25 05:38:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:38:32.586063 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37fb9300-769b-4a41-a649-363b1d101586 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:38:32.599575 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37fb9300-769b-4a41-a649-363b1d101586 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:38:32.607429 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb02273-bce2-4559-9e17-1e9f11b3d5a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:38:32.615708 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb02273-bce2-4559-9e17-1e9f11b3d5a5 admin admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:38:32.615708 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb02273-bce2-4559-9e17-1e9f11b3d5a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:38:32.615708 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb02273-bce2-4559-9e17-1e9f11b3d5a5 admin admin] PATH_INFO: `/v3/domains` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:38:32.615708 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cb02273-bce2-4559-9e17-1e9f11b3d5a5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:38:32.615708 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cb02273-bce2-4559-9e17-1e9f11b3d5a5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:38:32.615708 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cb02273-bce2-4559-9e17-1e9f11b3d5a5 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:38:32.640277 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1078/2155] 10.222.0.33 () {58 vars in 1152 bytes} [Mon Jul 25 05:38:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:38:32.646434 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dbf11bc-4118-421a-9b4f-3d668bbec188 None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:38:32.659627 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dbf11bc-4118-421a-9b4f-3d668bbec188 None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:38:32.666274 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f31134-160d-4d8e-9752-c922fd73f5e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:38:32.666993 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f31134-160d-4d8e-9752-c922fd73f5e2 admin admin] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:38:32.667701 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f31134-160d-4d8e-9752-c922fd73f5e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:38:32.668008 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f31134-160d-4d8e-9752-c922fd73f5e2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:38:32.669188 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21f31134-160d-4d8e-9752-c922fd73f5e2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:38:32.671131 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21f31134-160d-4d8e-9752-c922fd73f5e2 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:38:32.713149 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1078/2156] 10.222.0.33 () {60 vars in 1152 bytes} [Mon Jul 25 05:38:32 2022] POST /identity/v3/projects => generated 377 bytes in 69 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 05:38:32.718020 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb02273-bce2-4559-9e17-1e9f11b3d5a5 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:38:32.748018 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb02273-bce2-4559-9e17-1e9f11b3d5a5 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:38:32.763204 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f31134-160d-4d8e-9752-c922fd73f5e2 admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:38:32.764600 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-21f31134-160d-4d8e-9752-c922fd73f5e2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:38:32.853538 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-095945b9-9166-4890-9a97-62efc44bb9eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:38:32.854555 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095945b9-9166-4890-9a97-62efc44bb9eb admin admin] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:38:32.855092 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095945b9-9166-4890-9a97-62efc44bb9eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:38:32.855542 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095945b9-9166-4890-9a97-62efc44bb9eb admin admin] PATH_INFO: `/v3/users` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:38:32.856611 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-095945b9-9166-4890-9a97-62efc44bb9eb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:38:32.858841 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-095945b9-9166-4890-9a97-62efc44bb9eb admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:38:32.884139 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1079/2157] 10.222.0.33 () {60 vars in 1143 bytes} [Mon Jul 25 05:38:32 2022] POST /identity/v3/users => generated 289 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 05:38:32.892220 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-095945b9-9166-4890-9a97-62efc44bb9eb admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:38:32.912504 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-095945b9-9166-4890-9a97-62efc44bb9eb admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:38:32.919600 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c4adfa1-d433-40ef-824a-fae54b9a21a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:38:32.920286 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4adfa1-d433-40ef-824a-fae54b9a21a3 admin admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:38:32.920578 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4adfa1-d433-40ef-824a-fae54b9a21a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:38:32.920895 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4adfa1-d433-40ef-824a-fae54b9a21a3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:38:32.921651 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c4adfa1-d433-40ef-824a-fae54b9a21a3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:38:32.923281 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c4adfa1-d433-40ef-824a-fae54b9a21a3 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:38:32.938075 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1080/2158] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jul 25 05:38:32 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 05:38:32.939023 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c4adfa1-d433-40ef-824a-fae54b9a21a3 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:38:32.944095 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f31134-160d-4d8e-9752-c922fd73f5e2 admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:38:32.950374 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c4adfa1-d433-40ef-824a-fae54b9a21a3 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:38:32.952113 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e52a6d2-4284-43c5-957b-628054098f75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:38:32.953179 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e52a6d2-4284-43c5-957b-628054098f75 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:38:32.953467 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e52a6d2-4284-43c5-957b-628054098f75 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:38:32.953801 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e52a6d2-4284-43c5-957b-628054098f75 admin admin] PATH_INFO: `/v3/projects/4852ad6bb4a44edcbc2f797588d5aa84` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:38:32.955982 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e52a6d2-4284-43c5-957b-628054098f75 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4852ad6bb4a44edcbc2f797588d5aa84)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:38:32.957251 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b819f151-16fa-4a02-9ae3-bf43f5c9b676 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:38:32.958399 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e52a6d2-4284-43c5-957b-628054098f75 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:38:32.960745 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b819f151-16fa-4a02-9ae3-bf43f5c9b676 admin admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:38:32.961233 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b819f151-16fa-4a02-9ae3-bf43f5c9b676 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:38:32.961486 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b819f151-16fa-4a02-9ae3-bf43f5c9b676 admin admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:38:32.962729 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b819f151-16fa-4a02-9ae3-bf43f5c9b676 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:38:32.965888 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b819f151-16fa-4a02-9ae3-bf43f5c9b676 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:38:33.001639 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1081/2159] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jul 25 05:38:32 2022] GET /identity/v3/roles => generated 1420 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 05:38:33.003154 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b819f151-16fa-4a02-9ae3-bf43f5c9b676 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:38:33.016054 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b819f151-16fa-4a02-9ae3-bf43f5c9b676 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:38:33.023330 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5371d1db-18ef-42fb-8bab-e7b00268ee1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:38:33.024443 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5371d1db-18ef-42fb-8bab-e7b00268ee1f admin admin] REQUEST_METHOD: `PUT` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:38:33.024739 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5371d1db-18ef-42fb-8bab-e7b00268ee1f admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:38:33.025059 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5371d1db-18ef-42fb-8bab-e7b00268ee1f admin admin] PATH_INFO: `/v3/projects/f0b7d41e75df46f28cebff70c16d4750/users/c1ae82a1862048cb9038b7c76190da8a/roles/5074e74e35f54d4b9bb415c34999a82a` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:38:33.047744 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5371d1db-18ef-42fb-8bab-e7b00268ee1f admin admin] RBAC: Authorizing `identity:create_grant(project_id=f0b7d41e75df46f28cebff70c16d4750, user_id=c1ae82a1862048cb9038b7c76190da8a, role_id=5074e74e35f54d4b9bb415c34999a82a)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:38:33.051403 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5371d1db-18ef-42fb-8bab-e7b00268ee1f admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:38:33.053051 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-4e52a6d2-4284-43c5-957b-628054098f75 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4852ad6bb4a44edcbc2f797588d5aa84', 'request_id': 'req-4e52a6d2-4284-43c5-957b-628054098f75'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:38:33.061650 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-4e52a6d2-4284-43c5-957b-628054098f75 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4852ad6bb4a44edcbc2f797588d5aa84', 'request_id': 'req-4e52a6d2-4284-43c5-957b-628054098f75'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:38:33.068864 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-4e52a6d2-4284-43c5-957b-628054098f75 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4852ad6bb4a44edcbc2f797588d5aa84', 'request_id': 'req-4e52a6d2-4284-43c5-957b-628054098f75'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:38:33.072754 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1082/2160] 10.222.0.33 () {60 vars in 1482 bytes} [Mon Jul 25 05:38:33 2022] PUT /identity/v3/projects/f0b7d41e75df46f28cebff70c16d4750/users/c1ae82a1862048cb9038b7c76190da8a/roles/5074e74e35f54d4b9bb415c34999a82a => generated 0 bytes in 71 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 05:38:33.080178 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904da893-7c4f-46e9-bab9-38f24f50e4d4 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:38:33.080669 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904da893-7c4f-46e9-bab9-38f24f50e4d4 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:38:33.081178 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904da893-7c4f-46e9-bab9-38f24f50e4d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:38:33.088100 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-4e52a6d2-4284-43c5-957b-628054098f75 admin admin] The token cache is being invalidate because project 4852ad6bb4a44edcbc2f797588d5aa84 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80248) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 05:38:33.088558 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-4e52a6d2-4284-43c5-957b-628054098f75 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:38:33.089574 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-4e52a6d2-4284-43c5-957b-628054098f75 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:38:33.094246 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1079/2161] 10.222.0.33 () {58 vars in 1232 bytes} [Mon Jul 25 05:38:32 2022] DELETE /identity/v3/projects/4852ad6bb4a44edcbc2f797588d5aa84 => generated 0 bytes in 332 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 25 05:38:33.101244 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ea1d10-964e-40c8-9fbd-5bf3284e32c8 None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:38:33.102158 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ea1d10-964e-40c8-9fbd-5bf3284e32c8 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:38:33.102616 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ea1d10-964e-40c8-9fbd-5bf3284e32c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:38:33.109115 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-904da893-7c4f-46e9-bab9-38f24f50e4d4 None None] MFA Rules not processed for user `1eae540242d94c10b0ad33960a855f08`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:38:33.111175 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-904da893-7c4f-46e9-bab9-38f24f50e4d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:38:33.129816 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-e6ea1d10-964e-40c8-9fbd-5bf3284e32c8 None None] MFA Rules not processed for user `1eae540242d94c10b0ad33960a855f08`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:38:33.131747 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-e6ea1d10-964e-40c8-9fbd-5bf3284e32c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:38:33.162570 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:38:33.172462 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:38:33.192577 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1083/2162] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 25 05:38:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 05:38:33.201373 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-904da893-7c4f-46e9-bab9-38f24f50e4d4 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:38:33.202956 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-904da893-7c4f-46e9-bab9-38f24f50e4d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:38:33.226713 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1080/2163] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 25 05:38:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:38:33.237969 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa5b40a-8887-4369-b3b3-4b972c89e3f2 None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:38:33.238526 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa5b40a-8887-4369-b3b3-4b972c89e3f2 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:38:33.238709 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa5b40a-8887-4369-b3b3-4b972c89e3f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:38:33.277776 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-1fa5b40a-8887-4369-b3b3-4b972c89e3f2 None None] MFA Rules not processed for user `1eae540242d94c10b0ad33960a855f08`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:38:33.280906 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-904da893-7c4f-46e9-bab9-38f24f50e4d4 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:38:33.283751 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-1fa5b40a-8887-4369-b3b3-4b972c89e3f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:38:33.290187 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d306c9-451c-45bf-a90e-8a046f9b1cc0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b2931200f99e43d79a50a9f6e02738e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92afba266cce4d1bad4134c351e74be3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:38:33.290856 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d306c9-451c-45bf-a90e-8a046f9b1cc0 service neutron] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:38:33.291141 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d306c9-451c-45bf-a90e-8a046f9b1cc0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:38:33.291422 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d306c9-451c-45bf-a90e-8a046f9b1cc0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:38:33.304316 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15d306c9-451c-45bf-a90e-8a046f9b1cc0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:38:33.304949 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1081/2164] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 25 05:38:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 69 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:38:33.309940 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15d306c9-451c-45bf-a90e-8a046f9b1cc0 service neutron] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:38:33.311077 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa5b40a-8887-4369-b3b3-4b972c89e3f2 None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:38:33.319868 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1084/2165] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jul 25 05:38:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 05:38:33.328240 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa5b40a-8887-4369-b3b3-4b972c89e3f2 None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:38:33.335081 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6cd15ec-6a66-47fc-916c-1ba405a6f455 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:38:33.335768 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6cd15ec-6a66-47fc-916c-1ba405a6f455 admin admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:38:33.336051 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6cd15ec-6a66-47fc-916c-1ba405a6f455 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:38:33.336339 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6cd15ec-6a66-47fc-916c-1ba405a6f455 admin admin] PATH_INFO: `/v3/domains` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:38:33.337294 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6cd15ec-6a66-47fc-916c-1ba405a6f455 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:38:33.337617 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6cd15ec-6a66-47fc-916c-1ba405a6f455 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:38:33.339228 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6cd15ec-6a66-47fc-916c-1ba405a6f455 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:38:33.364638 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1082/2166] 10.222.0.33 () {58 vars in 1152 bytes} [Mon Jul 25 05:38:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:38:33.365695 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d306c9-451c-45bf-a90e-8a046f9b1cc0 service neutron] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:38:33.380013 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d306c9-451c-45bf-a90e-8a046f9b1cc0 service neutron] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:38:33.386007 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26842506-cc3c-4252-aade-b109e00224a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:38:33.386758 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26842506-cc3c-4252-aade-b109e00224a2 admin admin] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:38:33.387005 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26842506-cc3c-4252-aade-b109e00224a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:38:33.387362 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26842506-cc3c-4252-aade-b109e00224a2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:38:33.388516 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26842506-cc3c-4252-aade-b109e00224a2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:38:33.390849 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26842506-cc3c-4252-aade-b109e00224a2 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:38:33.452470 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1085/2167] 10.222.0.33 () {60 vars in 1152 bytes} [Mon Jul 25 05:38:33 2022] POST /identity/v3/projects => generated 367 bytes in 87 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 05:38:33.455189 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6cd15ec-6a66-47fc-916c-1ba405a6f455 admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:38:33.468211 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6cd15ec-6a66-47fc-916c-1ba405a6f455 admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:38:33.540342 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2128394-e0a7-4030-8597-9426d760b517 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:38:33.541602 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2128394-e0a7-4030-8597-9426d760b517 admin admin] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:38:33.542083 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2128394-e0a7-4030-8597-9426d760b517 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:38:33.542569 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2128394-e0a7-4030-8597-9426d760b517 admin admin] PATH_INFO: `/v3/users` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:38:33.543742 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2128394-e0a7-4030-8597-9426d760b517 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:38:33.546556 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2128394-e0a7-4030-8597-9426d760b517 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:38:33.573786 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26842506-cc3c-4252-aade-b109e00224a2 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:38:33.576375 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-26842506-cc3c-4252-aade-b109e00224a2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:38:33.587323 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1083/2168] 10.222.0.33 () {60 vars in 1143 bytes} [Mon Jul 25 05:38:33 2022] POST /identity/v3/users => generated 284 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 05:38:33.587323 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2128394-e0a7-4030-8597-9426d760b517 admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:38:33.600438 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2128394-e0a7-4030-8597-9426d760b517 admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:38:33.607762 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c3a070-59f8-435e-a011-7ec8264a2318 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:38:33.608527 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c3a070-59f8-435e-a011-7ec8264a2318 admin admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:38:33.609924 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c3a070-59f8-435e-a011-7ec8264a2318 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:38:33.609924 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c3a070-59f8-435e-a011-7ec8264a2318 admin admin] PATH_INFO: `/v3/roles` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:38:33.609924 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9c3a070-59f8-435e-a011-7ec8264a2318 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:38:33.612217 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9c3a070-59f8-435e-a011-7ec8264a2318 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:38:33.626003 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1084/2169] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jul 25 05:38:33 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 05:38:33.630781 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c3a070-59f8-435e-a011-7ec8264a2318 admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:38:33.654967 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c3a070-59f8-435e-a011-7ec8264a2318 admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:38:33.667577 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37146305-f13c-4469-b97b-170d9f2dba31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:38:33.669132 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37146305-f13c-4469-b97b-170d9f2dba31 admin admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:38:33.669843 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37146305-f13c-4469-b97b-170d9f2dba31 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:38:33.670245 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37146305-f13c-4469-b97b-170d9f2dba31 admin admin] PATH_INFO: `/v3/roles` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:38:33.672003 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37146305-f13c-4469-b97b-170d9f2dba31 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:38:33.675017 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37146305-f13c-4469-b97b-170d9f2dba31 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:38:33.676435 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26842506-cc3c-4252-aade-b109e00224a2 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:38:33.681345 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd30aa60-b84b-4ecc-a3c9-acba7e49ba2a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db57b478c25643dbaf936f90f6e17d0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92afba266cce4d1bad4134c351e74be3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:38:33.682037 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd30aa60-b84b-4ecc-a3c9-acba7e49ba2a service cinder] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:38:33.682403 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd30aa60-b84b-4ecc-a3c9-acba7e49ba2a service cinder] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:38:33.682651 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd30aa60-b84b-4ecc-a3c9-acba7e49ba2a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:38:33.685988 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-bd30aa60-b84b-4ecc-a3c9-acba7e49ba2a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:38:33.693300 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1085/2170] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jul 25 05:38:33 2022] GET /identity/v3/roles => generated 1420 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 05:38:33.698144 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37146305-f13c-4469-b97b-170d9f2dba31 admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:38:33.711532 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37146305-f13c-4469-b97b-170d9f2dba31 admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:38:33.721118 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c734834-5999-47a6-8929-846a07023f78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:38:33.723049 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c734834-5999-47a6-8929-846a07023f78 admin admin] REQUEST_METHOD: `PUT` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:38:33.723619 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c734834-5999-47a6-8929-846a07023f78 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:38:33.724209 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c734834-5999-47a6-8929-846a07023f78 admin admin] PATH_INFO: `/v3/projects/0f5638765ef9440d89e36b7613349c1e/users/322bd4bffdf6444d908560b4fd2b8184/roles/5074e74e35f54d4b9bb415c34999a82a` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:38:33.754227 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c734834-5999-47a6-8929-846a07023f78 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0f5638765ef9440d89e36b7613349c1e, user_id=322bd4bffdf6444d908560b4fd2b8184, role_id=5074e74e35f54d4b9bb415c34999a82a)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:38:33.757724 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c734834-5999-47a6-8929-846a07023f78 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:38:33.776640 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd30aa60-b84b-4ecc-a3c9-acba7e49ba2a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:38:33.778532 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd30aa60-b84b-4ecc-a3c9-acba7e49ba2a service cinder] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:38:33.779851 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1086/2171] 10.222.0.33 () {60 vars in 1482 bytes} [Mon Jul 25 05:38:33 2022] PUT /identity/v3/projects/0f5638765ef9440d89e36b7613349c1e/users/322bd4bffdf6444d908560b4fd2b8184/roles/5074e74e35f54d4b9bb415c34999a82a => generated 0 bytes in 83 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 05:38:33.790649 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7867b99c-5279-4679-ad78-cd0cdc6b4f2f None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:38:33.791407 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7867b99c-5279-4679-ad78-cd0cdc6b4f2f None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:38:33.791822 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7867b99c-5279-4679-ad78-cd0cdc6b4f2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:38:33.801582 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1086/2172] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jul 25 05:38:33 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 217 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 05:38:33.848259 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-7867b99c-5279-4679-ad78-cd0cdc6b4f2f None None] MFA Rules not processed for user `322bd4bffdf6444d908560b4fd2b8184`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:38:33.849544 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-7867b99c-5279-4679-ad78-cd0cdc6b4f2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:38:33.902652 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:38:33.944653 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1087/2173] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 25 05:38:33 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 05:38:33.948545 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd30aa60-b84b-4ecc-a3c9-acba7e49ba2a service cinder] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:38:33.962026 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd30aa60-b84b-4ecc-a3c9-acba7e49ba2a service cinder] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:38:33.994956 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:38:34.035919 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12728464-7749-460b-9f12-ee1ca9a7340b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:38:34.036991 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12728464-7749-460b-9f12-ee1ca9a7340b admin admin] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:38:34.037597 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12728464-7749-460b-9f12-ee1ca9a7340b admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:38:34.038052 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12728464-7749-460b-9f12-ee1ca9a7340b admin admin] PATH_INFO: `/v3/projects` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:38:34.038935 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12728464-7749-460b-9f12-ee1ca9a7340b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:38:34.041333 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12728464-7749-460b-9f12-ee1ca9a7340b admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:38:34.104177 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1087/2174] 10.222.0.33 () {60 vars in 1152 bytes} [Mon Jul 25 05:38:33 2022] POST /identity/v3/projects => generated 369 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 05:38:34.110402 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7867b99c-5279-4679-ad78-cd0cdc6b4f2f None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:38:34.126049 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7867b99c-5279-4679-ad78-cd0cdc6b4f2f None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:38:34.238464 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ab653d-e0a5-4270-9c58-e21974f5e607 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:38:34.239201 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ab653d-e0a5-4270-9c58-e21974f5e607 admin admin] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:38:34.239591 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ab653d-e0a5-4270-9c58-e21974f5e607 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:38:34.240453 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ab653d-e0a5-4270-9c58-e21974f5e607 admin admin] PATH_INFO: `/v3/users` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:38:34.241237 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01ab653d-e0a5-4270-9c58-e21974f5e607 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:38:34.244113 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01ab653d-e0a5-4270-9c58-e21974f5e607 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:38:34.281684 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1088/2175] 10.222.0.33 () {60 vars in 1143 bytes} [Mon Jul 25 05:38:34 2022] POST /identity/v3/users => generated 291 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 05:38:34.282820 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12728464-7749-460b-9f12-ee1ca9a7340b admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:38:34.297110 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12728464-7749-460b-9f12-ee1ca9a7340b admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:38:34.303227 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe6335e-7acb-498e-a340-b7ec4ab6844e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:38:34.304036 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe6335e-7acb-498e-a340-b7ec4ab6844e admin admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:38:34.304327 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe6335e-7acb-498e-a340-b7ec4ab6844e admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:38:34.304611 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe6335e-7acb-498e-a340-b7ec4ab6844e admin admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:38:34.305490 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbe6335e-7acb-498e-a340-b7ec4ab6844e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:38:34.307109 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbe6335e-7acb-498e-a340-b7ec4ab6844e admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:38:34.318778 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1088/2176] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jul 25 05:38:34 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 05:38:34.322824 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ab653d-e0a5-4270-9c58-e21974f5e607 admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:38:34.334535 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ab653d-e0a5-4270-9c58-e21974f5e607 admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:38:34.340103 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a21137-587d-4b44-a6a8-52a1035fbb09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:38:34.341568 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a21137-587d-4b44-a6a8-52a1035fbb09 admin admin] REQUEST_METHOD: `PUT` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:38:34.342049 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a21137-587d-4b44-a6a8-52a1035fbb09 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:38:34.342307 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a21137-587d-4b44-a6a8-52a1035fbb09 admin admin] PATH_INFO: `/v3/projects/4cca9815ca534f04bd785bb6be6f7be1/users/8badfca9621e46bbba085984fef135be/roles/39e5c666285a42fe8fc2c3afca6260d1` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:38:34.358997 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52a21137-587d-4b44-a6a8-52a1035fbb09 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4cca9815ca534f04bd785bb6be6f7be1, user_id=8badfca9621e46bbba085984fef135be, role_id=39e5c666285a42fe8fc2c3afca6260d1)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:38:34.361748 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52a21137-587d-4b44-a6a8-52a1035fbb09 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:38:34.377809 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1089/2177] 10.222.0.33 () {60 vars in 1482 bytes} [Mon Jul 25 05:38:34 2022] PUT /identity/v3/projects/4cca9815ca534f04bd785bb6be6f7be1/users/8badfca9621e46bbba085984fef135be/roles/39e5c666285a42fe8fc2c3afca6260d1 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 05:38:34.381604 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe6335e-7acb-498e-a340-b7ec4ab6844e admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:38:34.392885 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe6335e-7acb-498e-a340-b7ec4ab6844e admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:38:34.470963 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8cbebfd-8667-4a3a-a701-25549905ecdd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:38:34.472490 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8cbebfd-8667-4a3a-a701-25549905ecdd admin admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:38:34.472490 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8cbebfd-8667-4a3a-a701-25549905ecdd admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:38:34.472696 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8cbebfd-8667-4a3a-a701-25549905ecdd admin admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:38:34.473334 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8cbebfd-8667-4a3a-a701-25549905ecdd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:38:34.475834 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8cbebfd-8667-4a3a-a701-25549905ecdd admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:38:34.489713 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1089/2178] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jul 25 05:38:34 2022] GET /identity/v3/roles => generated 1420 bytes in 109 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 05:38:34.494166 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a21137-587d-4b44-a6a8-52a1035fbb09 admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:38:34.506348 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a21137-587d-4b44-a6a8-52a1035fbb09 admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:38:34.511082 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e983e0-38ec-47b8-adaa-6a23dfd40251 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:38:34.512099 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e983e0-38ec-47b8-adaa-6a23dfd40251 admin admin] REQUEST_METHOD: `PUT` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:38:34.513130 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e983e0-38ec-47b8-adaa-6a23dfd40251 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:38:34.513333 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e983e0-38ec-47b8-adaa-6a23dfd40251 admin admin] PATH_INFO: `/v3/projects/4cca9815ca534f04bd785bb6be6f7be1/users/8badfca9621e46bbba085984fef135be/roles/39e5c666285a42fe8fc2c3afca6260d1` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:38:34.515645 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0e983e0-38ec-47b8-adaa-6a23dfd40251 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4cca9815ca534f04bd785bb6be6f7be1, user_id=8badfca9621e46bbba085984fef135be, role_id=39e5c666285a42fe8fc2c3afca6260d1)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:38:34.518382 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0e983e0-38ec-47b8-adaa-6a23dfd40251 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:38:34.530836 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1090/2179] 10.222.0.33 () {60 vars in 1482 bytes} [Mon Jul 25 05:38:34 2022] PUT /identity/v3/projects/4cca9815ca534f04bd785bb6be6f7be1/users/8badfca9621e46bbba085984fef135be/roles/39e5c666285a42fe8fc2c3afca6260d1 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 05:38:34.546189 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719b16c1-a155-4131-afb3-5d7321f58a5c None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:38:34.546836 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719b16c1-a155-4131-afb3-5d7321f58a5c None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:38:34.547126 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719b16c1-a155-4131-afb3-5d7321f58a5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:38:34.583800 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-719b16c1-a155-4131-afb3-5d7321f58a5c None None] MFA Rules not processed for user `8badfca9621e46bbba085984fef135be`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:38:34.585971 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-719b16c1-a155-4131-afb3-5d7321f58a5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:38:34.663444 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1090/2180] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 25 05:38:34 2022] POST /identity/v3/auth/tokens => generated 3303 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:38:34.675828 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e983e0-38ec-47b8-adaa-6a23dfd40251 admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:38:34.678214 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-d0e983e0-38ec-47b8-adaa-6a23dfd40251 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:38:34.747886 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e983e0-38ec-47b8-adaa-6a23dfd40251 admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:38:34.752330 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db56d899-05f5-40ce-b5a3-9395e78c7faf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db57b478c25643dbaf936f90f6e17d0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92afba266cce4d1bad4134c351e74be3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:38:34.753046 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db56d899-05f5-40ce-b5a3-9395e78c7faf service cinder] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:38:34.753335 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db56d899-05f5-40ce-b5a3-9395e78c7faf service cinder] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:38:34.753620 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db56d899-05f5-40ce-b5a3-9395e78c7faf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:38:34.766277 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db56d899-05f5-40ce-b5a3-9395e78c7faf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:38:34.767680 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db56d899-05f5-40ce-b5a3-9395e78c7faf service cinder] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:38:34.779490 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1091/2181] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jul 25 05:38:34 2022] GET /identity/v3/auth/tokens => generated 3303 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 05:38:34.885988 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719b16c1-a155-4131-afb3-5d7321f58a5c None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:38:34.897705 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719b16c1-a155-4131-afb3-5d7321f58a5c None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:38:34.904557 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5228a893-3daf-4c68-92ce-213d9c4e2873 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db57b478c25643dbaf936f90f6e17d0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92afba266cce4d1bad4134c351e74be3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:38:34.905247 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5228a893-3daf-4c68-92ce-213d9c4e2873 service cinder] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:38:34.905534 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5228a893-3daf-4c68-92ce-213d9c4e2873 service cinder] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:38:34.905809 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5228a893-3daf-4c68-92ce-213d9c4e2873 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:38:34.933017 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:38:34.973700 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5228a893-3daf-4c68-92ce-213d9c4e2873 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:38:34.975281 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5228a893-3daf-4c68-92ce-213d9c4e2873 service cinder] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:38:34.985961 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1091/2182] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jul 25 05:38:34 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 05:38:35.823207 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db56d899-05f5-40ce-b5a3-9395e78c7faf service cinder] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:38:35.833794 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db56d899-05f5-40ce-b5a3-9395e78c7faf service cinder] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:38:35.838007 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c365bec-7d65-477a-bf30-5005657cb0bd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db57b478c25643dbaf936f90f6e17d0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92afba266cce4d1bad4134c351e74be3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:38:35.838756 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c365bec-7d65-477a-bf30-5005657cb0bd service cinder] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:38:35.839119 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c365bec-7d65-477a-bf30-5005657cb0bd service cinder] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:38:35.839474 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c365bec-7d65-477a-bf30-5005657cb0bd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:38:35.841653 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-8c365bec-7d65-477a-bf30-5005657cb0bd service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:38:35.878663 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:38:35.926909 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c365bec-7d65-477a-bf30-5005657cb0bd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:38:35.929680 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c365bec-7d65-477a-bf30-5005657cb0bd service cinder] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:38:35.946768 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1092/2183] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jul 25 05:38:35 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 25 05:38:38.461519 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c8a04e-ae08-4863-8488-ca7a4486b970 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:38:38.462043 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c8a04e-ae08-4863-8488-ca7a4486b970 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:38:38.462139 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c8a04e-ae08-4863-8488-ca7a4486b970 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:38:38.517203 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-b6c8a04e-ae08-4863-8488-ca7a4486b970 None None] MFA Rules not processed for user `c1ae82a1862048cb9038b7c76190da8a`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:38:38.518462 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-b6c8a04e-ae08-4863-8488-ca7a4486b970 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:38:38.641500 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1092/2184] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 25 05:38:38 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:38:38.657741 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c365bec-7d65-477a-bf30-5005657cb0bd service cinder] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:38:38.659311 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-8c365bec-7d65-477a-bf30-5005657cb0bd service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:38:38.802706 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c365bec-7d65-477a-bf30-5005657cb0bd service cinder] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:38:38.827762 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9840f914-bf95-4695-b412-f4bed1d7ff83 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cd482a61e8a420db45bb532e5dae6e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92afba266cce4d1bad4134c351e74be3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:38:38.828771 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9840f914-bf95-4695-b412-f4bed1d7ff83 service nova] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:38:38.829386 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9840f914-bf95-4695-b412-f4bed1d7ff83 service nova] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:38:38.829995 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9840f914-bf95-4695-b412-f4bed1d7ff83 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:38:38.843395 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9840f914-bf95-4695-b412-f4bed1d7ff83 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:38:38.845460 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9840f914-bf95-4695-b412-f4bed1d7ff83 service nova] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:38:38.866003 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1093/2185] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jul 25 05:38:38 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 209 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 05:38:39.439103 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c8a04e-ae08-4863-8488-ca7a4486b970 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:38:39.542489 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c8a04e-ae08-4863-8488-ca7a4486b970 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:38:39.548028 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f2f8ac-b46c-4d4e-ab2e-ff4cfd377a9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:38:39.548955 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f2f8ac-b46c-4d4e-ab2e-ff4cfd377a9d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:38:39.549246 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f2f8ac-b46c-4d4e-ab2e-ff4cfd377a9d admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:38:39.549529 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f2f8ac-b46c-4d4e-ab2e-ff4cfd377a9d admin admin] PATH_INFO: `/v3/users/322bd4bffdf6444d908560b4fd2b8184` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:38:39.550950 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63f2f8ac-b46c-4d4e-ab2e-ff4cfd377a9d admin admin] RBAC: Authorizing `identity:delete_user(user_id=322bd4bffdf6444d908560b4fd2b8184)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:38:39.552995 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63f2f8ac-b46c-4d4e-ab2e-ff4cfd377a9d admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:38:39.610394 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-63f2f8ac-b46c-4d4e-ab2e-ff4cfd377a9d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '322bd4bffdf6444d908560b4fd2b8184'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:38:39.611106 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-63f2f8ac-b46c-4d4e-ab2e-ff4cfd377a9d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '322bd4bffdf6444d908560b4fd2b8184'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:38:39.620740 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-63f2f8ac-b46c-4d4e-ab2e-ff4cfd377a9d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '322bd4bffdf6444d908560b4fd2b8184'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:38:39.623366 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:38:39.643559 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-63f2f8ac-b46c-4d4e-ab2e-ff4cfd377a9d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '322bd4bffdf6444d908560b4fd2b8184'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:38:39.644902 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-63f2f8ac-b46c-4d4e-ab2e-ff4cfd377a9d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '322bd4bffdf6444d908560b4fd2b8184'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:38:39.672081 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1093/2186] 10.222.0.33 () {58 vars in 1223 bytes} [Mon Jul 25 05:38:39 2022] DELETE /identity/v3/users/322bd4bffdf6444d908560b4fd2b8184 => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 05:38:39.674162 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db17183f-f045-4d87-b26a-5f6f090cffb2 None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:38:39.675110 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db17183f-f045-4d87-b26a-5f6f090cffb2 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:38:39.675766 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db17183f-f045-4d87-b26a-5f6f090cffb2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:38:39.702005 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-db17183f-f045-4d87-b26a-5f6f090cffb2 None None] MFA Rules not processed for user `1eae540242d94c10b0ad33960a855f08`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:38:39.703744 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-db17183f-f045-4d87-b26a-5f6f090cffb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:38:39.771906 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1094/2187] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 25 05:38:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:38:39.782414 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f2f8ac-b46c-4d4e-ab2e-ff4cfd377a9d admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:38:39.783899 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-63f2f8ac-b46c-4d4e-ab2e-ff4cfd377a9d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:38:39.863011 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f2f8ac-b46c-4d4e-ab2e-ff4cfd377a9d admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:38:39.868170 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4810e6a5-a570-4fa7-a34f-4a6cf61ddaa0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b2931200f99e43d79a50a9f6e02738e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92afba266cce4d1bad4134c351e74be3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:38:39.869266 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4810e6a5-a570-4fa7-a34f-4a6cf61ddaa0 service neutron] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:38:39.870022 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4810e6a5-a570-4fa7-a34f-4a6cf61ddaa0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:38:39.870280 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4810e6a5-a570-4fa7-a34f-4a6cf61ddaa0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:38:39.880948 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4810e6a5-a570-4fa7-a34f-4a6cf61ddaa0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:38:39.882496 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4810e6a5-a570-4fa7-a34f-4a6cf61ddaa0 service neutron] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:38:39.897210 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1094/2188] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jul 25 05:38:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 05:38:40.200287 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db17183f-f045-4d87-b26a-5f6f090cffb2 None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:38:40.202169 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-db17183f-f045-4d87-b26a-5f6f090cffb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:38:40.216235 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db17183f-f045-4d87-b26a-5f6f090cffb2 None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:38:40.221451 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f63314-c05d-4aaa-8953-1699fd53d8fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:38:40.222377 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f63314-c05d-4aaa-8953-1699fd53d8fe admin admin] REQUEST_METHOD: `DELETE` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:38:40.222829 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f63314-c05d-4aaa-8953-1699fd53d8fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:38:40.223102 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f63314-c05d-4aaa-8953-1699fd53d8fe admin admin] PATH_INFO: `/v3/projects/0f5638765ef9440d89e36b7613349c1e` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:38:40.224487 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4f63314-c05d-4aaa-8953-1699fd53d8fe admin admin] RBAC: Authorizing `identity:delete_project(project_id=0f5638765ef9440d89e36b7613349c1e)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:38:40.226424 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4f63314-c05d-4aaa-8953-1699fd53d8fe admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:38:40.273911 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:38:40.275599 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-a4f63314-c05d-4aaa-8953-1699fd53d8fe admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0f5638765ef9440d89e36b7613349c1e', 'request_id': 'req-a4f63314-c05d-4aaa-8953-1699fd53d8fe'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:38:40.283079 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-a4f63314-c05d-4aaa-8953-1699fd53d8fe admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0f5638765ef9440d89e36b7613349c1e', 'request_id': 'req-a4f63314-c05d-4aaa-8953-1699fd53d8fe'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:38:40.289429 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-a4f63314-c05d-4aaa-8953-1699fd53d8fe admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0f5638765ef9440d89e36b7613349c1e', 'request_id': 'req-a4f63314-c05d-4aaa-8953-1699fd53d8fe'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:38:40.302427 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-a4f63314-c05d-4aaa-8953-1699fd53d8fe admin admin] The token cache is being invalidate because project 0f5638765ef9440d89e36b7613349c1e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80248) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 05:38:40.302721 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-a4f63314-c05d-4aaa-8953-1699fd53d8fe admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:38:40.303475 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-a4f63314-c05d-4aaa-8953-1699fd53d8fe admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:38:40.305139 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1095/2189] 10.222.0.33 () {58 vars in 1232 bytes} [Mon Jul 25 05:38:40 2022] DELETE /identity/v3/projects/0f5638765ef9440d89e36b7613349c1e => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 05:38:40.309717 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4810e6a5-a570-4fa7-a34f-4a6cf61ddaa0 service neutron] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:38:40.319993 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4810e6a5-a570-4fa7-a34f-4a6cf61ddaa0 service neutron] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:38:40.390687 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e64b3f8-b86b-4d93-a7f4-5fed57023173 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:38:40.391580 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e64b3f8-b86b-4d93-a7f4-5fed57023173 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:38:40.391864 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e64b3f8-b86b-4d93-a7f4-5fed57023173 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:38:40.392138 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e64b3f8-b86b-4d93-a7f4-5fed57023173 admin admin] PATH_INFO: `/v3/users/8badfca9621e46bbba085984fef135be` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:38:40.393571 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e64b3f8-b86b-4d93-a7f4-5fed57023173 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8badfca9621e46bbba085984fef135be)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:38:40.395639 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e64b3f8-b86b-4d93-a7f4-5fed57023173 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:38:40.469867 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-3e64b3f8-b86b-4d93-a7f4-5fed57023173 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8badfca9621e46bbba085984fef135be'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:38:40.470751 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-3e64b3f8-b86b-4d93-a7f4-5fed57023173 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8badfca9621e46bbba085984fef135be'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:38:40.483417 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-3e64b3f8-b86b-4d93-a7f4-5fed57023173 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8badfca9621e46bbba085984fef135be'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:38:40.498841 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-3e64b3f8-b86b-4d93-a7f4-5fed57023173 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8badfca9621e46bbba085984fef135be'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:38:40.499711 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-3e64b3f8-b86b-4d93-a7f4-5fed57023173 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8badfca9621e46bbba085984fef135be'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:38:40.503855 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:38:40.540016 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1095/2190] 10.222.0.33 () {58 vars in 1223 bytes} [Mon Jul 25 05:38:40 2022] DELETE /identity/v3/users/8badfca9621e46bbba085984fef135be => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 05:38:40.605243 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7dae9d-4d3c-4bd3-bead-8a2f410ed18a None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:38:40.605923 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7dae9d-4d3c-4bd3-bead-8a2f410ed18a None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:38:40.605923 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7dae9d-4d3c-4bd3-bead-8a2f410ed18a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:38:40.625914 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-fd7dae9d-4d3c-4bd3-bead-8a2f410ed18a None None] MFA Rules not processed for user `c1ae82a1862048cb9038b7c76190da8a`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:38:40.627817 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-fd7dae9d-4d3c-4bd3-bead-8a2f410ed18a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:38:40.717966 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1096/2191] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 25 05:38:40 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:38:40.905520 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e64b3f8-b86b-4d93-a7f4-5fed57023173 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:38:40.907130 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-3e64b3f8-b86b-4d93-a7f4-5fed57023173 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:38:40.993569 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e64b3f8-b86b-4d93-a7f4-5fed57023173 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:38:40.999685 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a17756b2-ed32-4014-9e03-4be0b57ed4e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:38:41.000557 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17756b2-ed32-4014-9e03-4be0b57ed4e8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:38:41.000876 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17756b2-ed32-4014-9e03-4be0b57ed4e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:38:41.001154 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17756b2-ed32-4014-9e03-4be0b57ed4e8 admin admin] PATH_INFO: `/v3/projects/4cca9815ca534f04bd785bb6be6f7be1` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:38:41.002357 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a17756b2-ed32-4014-9e03-4be0b57ed4e8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4cca9815ca534f04bd785bb6be6f7be1)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:38:41.004257 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a17756b2-ed32-4014-9e03-4be0b57ed4e8 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:38:41.053268 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-a17756b2-ed32-4014-9e03-4be0b57ed4e8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4cca9815ca534f04bd785bb6be6f7be1', 'request_id': 'req-a17756b2-ed32-4014-9e03-4be0b57ed4e8'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:38:41.058979 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-a17756b2-ed32-4014-9e03-4be0b57ed4e8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4cca9815ca534f04bd785bb6be6f7be1', 'request_id': 'req-a17756b2-ed32-4014-9e03-4be0b57ed4e8'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:38:41.065241 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-a17756b2-ed32-4014-9e03-4be0b57ed4e8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4cca9815ca534f04bd785bb6be6f7be1', 'request_id': 'req-a17756b2-ed32-4014-9e03-4be0b57ed4e8'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:38:41.078442 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-a17756b2-ed32-4014-9e03-4be0b57ed4e8 admin admin] The token cache is being invalidate because project 4cca9815ca534f04bd785bb6be6f7be1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80249) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 05:38:41.078737 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-a17756b2-ed32-4014-9e03-4be0b57ed4e8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:38:41.079396 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-a17756b2-ed32-4014-9e03-4be0b57ed4e8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:38:41.080983 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1096/2192] 10.222.0.33 () {58 vars in 1232 bytes} [Mon Jul 25 05:38:40 2022] DELETE /identity/v3/projects/4cca9815ca534f04bd785bb6be6f7be1 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 05:38:41.086081 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd02df2a-7e00-48e7-8cf7-56a712e59cc9 None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:38:41.086382 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd02df2a-7e00-48e7-8cf7-56a712e59cc9 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:38:41.086671 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd02df2a-7e00-48e7-8cf7-56a712e59cc9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:38:41.105664 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-dd02df2a-7e00-48e7-8cf7-56a712e59cc9 None None] MFA Rules not processed for user `1eae540242d94c10b0ad33960a855f08`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:38:41.107460 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-dd02df2a-7e00-48e7-8cf7-56a712e59cc9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:38:41.175350 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1097/2193] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 25 05:38:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:38:41.183109 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73cc9d64-c7c2-4032-9a3e-4504fd932342 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:38:41.183401 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73cc9d64-c7c2-4032-9a3e-4504fd932342 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:38:41.183678 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73cc9d64-c7c2-4032-9a3e-4504fd932342 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:38:41.202376 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-73cc9d64-c7c2-4032-9a3e-4504fd932342 None None] MFA Rules not processed for user `1eae540242d94c10b0ad33960a855f08`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:38:41.203592 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-73cc9d64-c7c2-4032-9a3e-4504fd932342 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:38:41.214013 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1097/2194] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 25 05:38:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:38:41.218809 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd02df2a-7e00-48e7-8cf7-56a712e59cc9 None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:38:41.229491 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd02df2a-7e00-48e7-8cf7-56a712e59cc9 None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:38:41.234319 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51831dd4-e7b9-486d-8cca-7e4bb78cd73c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:38:41.234983 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51831dd4-e7b9-486d-8cca-7e4bb78cd73c admin admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:38:41.235265 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51831dd4-e7b9-486d-8cca-7e4bb78cd73c admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:38:41.235707 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51831dd4-e7b9-486d-8cca-7e4bb78cd73c admin admin] PATH_INFO: `/v3/domains` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:38:41.236432 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51831dd4-e7b9-486d-8cca-7e4bb78cd73c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:38:41.236935 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51831dd4-e7b9-486d-8cca-7e4bb78cd73c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:38:41.238372 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51831dd4-e7b9-486d-8cca-7e4bb78cd73c admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:38:41.252948 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1098/2195] 10.222.0.33 () {58 vars in 1152 bytes} [Mon Jul 25 05:38:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:38:41.256946 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73cc9d64-c7c2-4032-9a3e-4504fd932342 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:38:41.268200 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73cc9d64-c7c2-4032-9a3e-4504fd932342 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:38:41.272939 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc6c484-1cc1-4f0c-badf-e918e2e7410c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:38:41.273592 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc6c484-1cc1-4f0c-badf-e918e2e7410c admin admin] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:38:41.273876 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc6c484-1cc1-4f0c-badf-e918e2e7410c admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:38:41.274151 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc6c484-1cc1-4f0c-badf-e918e2e7410c admin admin] PATH_INFO: `/v3/projects` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:38:41.274869 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dc6c484-1cc1-4f0c-badf-e918e2e7410c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:38:41.276743 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dc6c484-1cc1-4f0c-badf-e918e2e7410c admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:38:41.305834 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1098/2196] 10.222.0.33 () {60 vars in 1152 bytes} [Mon Jul 25 05:38:41 2022] POST /identity/v3/projects => generated 401 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 05:38:41.307004 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51831dd4-e7b9-486d-8cca-7e4bb78cd73c admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:38:41.318168 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51831dd4-e7b9-486d-8cca-7e4bb78cd73c admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:38:41.380716 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bae5d98-e5e4-4fde-836e-865294d34be4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:38:41.381506 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bae5d98-e5e4-4fde-836e-865294d34be4 admin admin] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:38:41.381796 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bae5d98-e5e4-4fde-836e-865294d34be4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:38:41.382074 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bae5d98-e5e4-4fde-836e-865294d34be4 admin admin] PATH_INFO: `/v3/users` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:38:41.382797 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bae5d98-e5e4-4fde-836e-865294d34be4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:38:41.384880 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bae5d98-e5e4-4fde-836e-865294d34be4 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:38:41.400284 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=93,keystone:DELETE=3,keystone:INSERT=4 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:38:41.409107 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1099/2197] 10.222.0.33 () {60 vars in 1143 bytes} [Mon Jul 25 05:38:41 2022] POST /identity/v3/users => generated 301 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 05:38:41.413068 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc6c484-1cc1-4f0c-badf-e918e2e7410c admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:38:41.425665 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc6c484-1cc1-4f0c-badf-e918e2e7410c admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:38:41.430984 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ceb006-de2b-4625-8d0f-a8dd447e136d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:38:41.431808 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ceb006-de2b-4625-8d0f-a8dd447e136d admin admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:38:41.432095 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ceb006-de2b-4625-8d0f-a8dd447e136d admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:38:41.432381 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ceb006-de2b-4625-8d0f-a8dd447e136d admin admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:38:41.433322 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0ceb006-de2b-4625-8d0f-a8dd447e136d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:38:41.435138 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0ceb006-de2b-4625-8d0f-a8dd447e136d admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:38:41.450582 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1099/2198] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jul 25 05:38:41 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 05:38:41.451844 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bae5d98-e5e4-4fde-836e-865294d34be4 admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:38:41.463316 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bae5d98-e5e4-4fde-836e-865294d34be4 admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:38:41.470507 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f12fec-a265-4596-9db9-857b874bc2c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:38:41.471291 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f12fec-a265-4596-9db9-857b874bc2c4 admin admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:38:41.471707 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f12fec-a265-4596-9db9-857b874bc2c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:38:41.471994 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f12fec-a265-4596-9db9-857b874bc2c4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:38:41.472768 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7f12fec-a265-4596-9db9-857b874bc2c4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:38:41.474405 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7f12fec-a265-4596-9db9-857b874bc2c4 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:38:41.486280 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1100/2199] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jul 25 05:38:41 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 25 05:38:41.490464 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ceb006-de2b-4625-8d0f-a8dd447e136d admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:38:41.502159 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ceb006-de2b-4625-8d0f-a8dd447e136d admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:38:41.510094 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e14e730e-3595-4981-953f-8e4ed2b50ed7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:38:41.511337 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14e730e-3595-4981-953f-8e4ed2b50ed7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:38:41.511614 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14e730e-3595-4981-953f-8e4ed2b50ed7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:38:41.511990 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14e730e-3595-4981-953f-8e4ed2b50ed7 admin admin] PATH_INFO: `/v3/projects/63dbaa5c92514af9bd6d74576a5b2147/users/109d2fe55a8748aea6df5b0efdf18eb6/roles/5074e74e35f54d4b9bb415c34999a82a` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:38:41.530819 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e14e730e-3595-4981-953f-8e4ed2b50ed7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=63dbaa5c92514af9bd6d74576a5b2147, user_id=109d2fe55a8748aea6df5b0efdf18eb6, role_id=5074e74e35f54d4b9bb415c34999a82a)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:38:41.533533 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e14e730e-3595-4981-953f-8e4ed2b50ed7 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:38:41.547588 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1100/2200] 10.222.0.33 () {60 vars in 1482 bytes} [Mon Jul 25 05:38:41 2022] PUT /identity/v3/projects/63dbaa5c92514af9bd6d74576a5b2147/users/109d2fe55a8748aea6df5b0efdf18eb6/roles/5074e74e35f54d4b9bb415c34999a82a => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 05:38:41.559332 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8aef96-d4da-4efa-8ed1-4f85eb7f81d2 None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:38:41.559768 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8aef96-d4da-4efa-8ed1-4f85eb7f81d2 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:38:41.560046 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8aef96-d4da-4efa-8ed1-4f85eb7f81d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:38:41.593479 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-5b8aef96-d4da-4efa-8ed1-4f85eb7f81d2 None None] MFA Rules not processed for user `109d2fe55a8748aea6df5b0efdf18eb6`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:38:41.595158 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-5b8aef96-d4da-4efa-8ed1-4f85eb7f81d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:38:41.690983 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1101/2201] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 25 05:38:41 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 05:38:41.695886 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e14e730e-3595-4981-953f-8e4ed2b50ed7 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:38:41.727066 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:38:41.782612 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e14e730e-3595-4981-953f-8e4ed2b50ed7 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:38:41.788624 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82eca307-9ccc-4bf9-9e5b-7829b19d39d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:38:41.789438 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82eca307-9ccc-4bf9-9e5b-7829b19d39d5 admin admin] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:38:41.789724 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82eca307-9ccc-4bf9-9e5b-7829b19d39d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:38:41.790006 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82eca307-9ccc-4bf9-9e5b-7829b19d39d5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:38:41.790937 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82eca307-9ccc-4bf9-9e5b-7829b19d39d5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:38:41.793307 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82eca307-9ccc-4bf9-9e5b-7829b19d39d5 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:38:41.852199 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1101/2202] 10.222.0.33 () {60 vars in 1152 bytes} [Mon Jul 25 05:38:41 2022] POST /identity/v3/projects => generated 399 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 25 05:38:41.856126 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b8aef96-d4da-4efa-8ed1-4f85eb7f81d2 None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:38:41.896888 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b8aef96-d4da-4efa-8ed1-4f85eb7f81d2 None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:38:42.032566 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1ac718-112d-49c5-b834-cd6a56102adf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:38:42.034556 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1ac718-112d-49c5-b834-cd6a56102adf admin admin] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:38:42.035638 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1ac718-112d-49c5-b834-cd6a56102adf admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:38:42.036882 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1ac718-112d-49c5-b834-cd6a56102adf admin admin] PATH_INFO: `/v3/users` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:38:42.037911 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d1ac718-112d-49c5-b834-cd6a56102adf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:38:42.040607 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d1ac718-112d-49c5-b834-cd6a56102adf admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:38:42.076808 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1102/2203] 10.222.0.33 () {60 vars in 1143 bytes} [Mon Jul 25 05:38:41 2022] POST /identity/v3/users => generated 306 bytes in 221 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 05:38:42.080737 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82eca307-9ccc-4bf9-9e5b-7829b19d39d5 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:38:42.096060 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82eca307-9ccc-4bf9-9e5b-7829b19d39d5 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:38:42.101294 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a4e1f1-32fb-420c-a0c7-e877941cd0a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:38:42.102281 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a4e1f1-32fb-420c-a0c7-e877941cd0a9 admin admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:38:42.102835 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a4e1f1-32fb-420c-a0c7-e877941cd0a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:38:42.103278 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a4e1f1-32fb-420c-a0c7-e877941cd0a9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:38:42.104128 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59a4e1f1-32fb-420c-a0c7-e877941cd0a9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:38:42.106528 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59a4e1f1-32fb-420c-a0c7-e877941cd0a9 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:38:42.130574 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1102/2204] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jul 25 05:38:42 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 25 05:38:42.134848 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1ac718-112d-49c5-b834-cd6a56102adf admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:38:42.149428 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1ac718-112d-49c5-b834-cd6a56102adf admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:38:42.159500 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b3b8040-de15-4498-9377-57f92c2c1481 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:38:42.160439 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3b8040-de15-4498-9377-57f92c2c1481 admin admin] REQUEST_METHOD: `PUT` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:38:42.160675 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3b8040-de15-4498-9377-57f92c2c1481 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:38:42.161403 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3b8040-de15-4498-9377-57f92c2c1481 admin admin] PATH_INFO: `/v3/projects/a91f41ec2e2746e9a4ecf04cf091bc2a/users/4a5c272f9428406eb7ea2f8d1cedce5a/roles/39e5c666285a42fe8fc2c3afca6260d1` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:38:42.186864 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b3b8040-de15-4498-9377-57f92c2c1481 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a91f41ec2e2746e9a4ecf04cf091bc2a, user_id=4a5c272f9428406eb7ea2f8d1cedce5a, role_id=39e5c666285a42fe8fc2c3afca6260d1)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:38:42.190014 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b3b8040-de15-4498-9377-57f92c2c1481 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:38:42.214248 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1103/2205] 10.222.0.33 () {60 vars in 1482 bytes} [Mon Jul 25 05:38:42 2022] PUT /identity/v3/projects/a91f41ec2e2746e9a4ecf04cf091bc2a/users/4a5c272f9428406eb7ea2f8d1cedce5a/roles/39e5c666285a42fe8fc2c3afca6260d1 => generated 0 bytes in 80 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 05:38:42.218716 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a4e1f1-32fb-420c-a0c7-e877941cd0a9 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:38:42.233401 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a4e1f1-32fb-420c-a0c7-e877941cd0a9 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:38:42.316641 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d32e1b-3f2e-4204-993b-8a1d6862f8c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:38:42.317360 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d32e1b-3f2e-4204-993b-8a1d6862f8c8 admin admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:38:42.317650 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d32e1b-3f2e-4204-993b-8a1d6862f8c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:38:42.317929 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d32e1b-3f2e-4204-993b-8a1d6862f8c8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:38:42.319056 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19d32e1b-3f2e-4204-993b-8a1d6862f8c8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:38:42.320490 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19d32e1b-3f2e-4204-993b-8a1d6862f8c8 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:38:42.338433 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1103/2206] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jul 25 05:38:42 2022] GET /identity/v3/roles => generated 1420 bytes in 120 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 05:38:42.339487 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b3b8040-de15-4498-9377-57f92c2c1481 admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:38:42.359520 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b3b8040-de15-4498-9377-57f92c2c1481 admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:38:42.364113 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-236f6951-f25e-4e16-97dd-193f5afdecfb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:38:42.365150 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236f6951-f25e-4e16-97dd-193f5afdecfb admin admin] REQUEST_METHOD: `PUT` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:38:42.365345 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236f6951-f25e-4e16-97dd-193f5afdecfb admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:38:42.365621 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236f6951-f25e-4e16-97dd-193f5afdecfb admin admin] PATH_INFO: `/v3/projects/a91f41ec2e2746e9a4ecf04cf091bc2a/users/4a5c272f9428406eb7ea2f8d1cedce5a/roles/39e5c666285a42fe8fc2c3afca6260d1` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:38:42.367733 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-236f6951-f25e-4e16-97dd-193f5afdecfb admin admin] RBAC: Authorizing `identity:create_grant(project_id=a91f41ec2e2746e9a4ecf04cf091bc2a, user_id=4a5c272f9428406eb7ea2f8d1cedce5a, role_id=39e5c666285a42fe8fc2c3afca6260d1)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:38:42.370406 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-236f6951-f25e-4e16-97dd-193f5afdecfb admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:38:42.391087 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1104/2207] 10.222.0.33 () {60 vars in 1482 bytes} [Mon Jul 25 05:38:42 2022] PUT /identity/v3/projects/a91f41ec2e2746e9a4ecf04cf091bc2a/users/4a5c272f9428406eb7ea2f8d1cedce5a/roles/39e5c666285a42fe8fc2c3afca6260d1 => generated 0 bytes in 43 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 25 05:38:42.392665 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b8e7e1-5239-4e3e-a699-8b253b52d172 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:38:42.393011 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b8e7e1-5239-4e3e-a699-8b253b52d172 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:38:42.393295 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b8e7e1-5239-4e3e-a699-8b253b52d172 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:38:42.437159 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-16b8e7e1-5239-4e3e-a699-8b253b52d172 None None] MFA Rules not processed for user `4a5c272f9428406eb7ea2f8d1cedce5a`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:38:42.439731 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-16b8e7e1-5239-4e3e-a699-8b253b52d172 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:38:42.580899 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1104/2208] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 25 05:38:42 2022] POST /identity/v3/auth/tokens => generated 3333 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 05:38:42.588516 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-236f6951-f25e-4e16-97dd-193f5afdecfb admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:38:42.592473 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-236f6951-f25e-4e16-97dd-193f5afdecfb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:38:42.663408 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:38:42.716600 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-236f6951-f25e-4e16-97dd-193f5afdecfb admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:38:42.725447 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f44b8a78-a257-4795-ad2f-68bafaf7decf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db57b478c25643dbaf936f90f6e17d0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92afba266cce4d1bad4134c351e74be3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:38:42.726127 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44b8a78-a257-4795-ad2f-68bafaf7decf service cinder] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:38:42.726473 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44b8a78-a257-4795-ad2f-68bafaf7decf service cinder] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:38:42.726799 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44b8a78-a257-4795-ad2f-68bafaf7decf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:38:42.739523 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f44b8a78-a257-4795-ad2f-68bafaf7decf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:38:42.741007 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f44b8a78-a257-4795-ad2f-68bafaf7decf service cinder] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:38:42.753383 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1105/2209] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jul 25 05:38:42 2022] GET /identity/v3/auth/tokens => generated 3333 bytes in 166 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 25 05:38:42.775843 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b8e7e1-5239-4e3e-a699-8b253b52d172 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:38:42.777328 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-16b8e7e1-5239-4e3e-a699-8b253b52d172 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:38:42.847949 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:38:42.866151 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f44b8a78-a257-4795-ad2f-68bafaf7decf service cinder] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:38:42.876295 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f44b8a78-a257-4795-ad2f-68bafaf7decf service cinder] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:38:42.883465 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a6ccf74-39e3-47f1-a078-81bb5af44f45 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db57b478c25643dbaf936f90f6e17d0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92afba266cce4d1bad4134c351e74be3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:38:42.884269 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6ccf74-39e3-47f1-a078-81bb5af44f45 service cinder] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:38:42.886005 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6ccf74-39e3-47f1-a078-81bb5af44f45 service cinder] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:38:42.886911 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6ccf74-39e3-47f1-a078-81bb5af44f45 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:38:42.954216 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b8e7e1-5239-4e3e-a699-8b253b52d172 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:38:42.967045 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ce4daa-fcd3-4aa1-aab0-24b43204681b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:38:42.968301 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ce4daa-fcd3-4aa1-aab0-24b43204681b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:38:42.969628 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ce4daa-fcd3-4aa1-aab0-24b43204681b admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:38:42.971227 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ce4daa-fcd3-4aa1-aab0-24b43204681b admin admin] PATH_INFO: `/v3/users/a537b33ba94140b6910a738a4028d2a1` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:38:42.973835 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26ce4daa-fcd3-4aa1-aab0-24b43204681b admin admin] RBAC: Authorizing `identity:delete_user(user_id=a537b33ba94140b6910a738a4028d2a1)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:38:42.976346 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26ce4daa-fcd3-4aa1-aab0-24b43204681b admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:38:43.030693 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a6ccf74-39e3-47f1-a078-81bb5af44f45 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:38:43.032523 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a6ccf74-39e3-47f1-a078-81bb5af44f45 service cinder] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:38:43.052014 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1106/2210] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jul 25 05:38:42 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 186 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 25 05:38:43.067859 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-26ce4daa-fcd3-4aa1-aab0-24b43204681b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a537b33ba94140b6910a738a4028d2a1'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:38:43.068939 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-26ce4daa-fcd3-4aa1-aab0-24b43204681b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a537b33ba94140b6910a738a4028d2a1'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:38:43.078843 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-26ce4daa-fcd3-4aa1-aab0-24b43204681b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a537b33ba94140b6910a738a4028d2a1'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:38:43.094307 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-26ce4daa-fcd3-4aa1-aab0-24b43204681b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a537b33ba94140b6910a738a4028d2a1'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:38:43.095226 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-26ce4daa-fcd3-4aa1-aab0-24b43204681b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a537b33ba94140b6910a738a4028d2a1'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:38:43.133053 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1105/2211] 10.222.0.33 () {58 vars in 1223 bytes} [Mon Jul 25 05:38:42 2022] DELETE /identity/v3/users/a537b33ba94140b6910a738a4028d2a1 => generated 0 bytes in 358 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 05:38:43.138033 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b88f17-516a-4d9c-9afb-0256dc133767 None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:38:43.138329 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b88f17-516a-4d9c-9afb-0256dc133767 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:38:43.138670 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b88f17-516a-4d9c-9afb-0256dc133767 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:38:43.174624 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-30b88f17-516a-4d9c-9afb-0256dc133767 None None] MFA Rules not processed for user `1eae540242d94c10b0ad33960a855f08`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:38:43.175678 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-30b88f17-516a-4d9c-9afb-0256dc133767 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:38:43.290126 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1107/2212] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 25 05:38:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:38:43.301479 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ce4daa-fcd3-4aa1-aab0-24b43204681b admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:38:43.306307 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-26ce4daa-fcd3-4aa1-aab0-24b43204681b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:38:43.419078 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ce4daa-fcd3-4aa1-aab0-24b43204681b admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:38:43.423604 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f6f310-cb44-4705-87d8-490978e9f6d5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b2931200f99e43d79a50a9f6e02738e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92afba266cce4d1bad4134c351e74be3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:38:43.424337 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f6f310-cb44-4705-87d8-490978e9f6d5 service neutron] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:38:43.424705 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f6f310-cb44-4705-87d8-490978e9f6d5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:38:43.425041 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f6f310-cb44-4705-87d8-490978e9f6d5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:38:43.437689 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5f6f310-cb44-4705-87d8-490978e9f6d5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:38:43.439240 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5f6f310-cb44-4705-87d8-490978e9f6d5 service neutron] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:38:43.457048 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1106/2213] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jul 25 05:38:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 156 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 05:38:43.861706 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b88f17-516a-4d9c-9afb-0256dc133767 None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:38:43.863225 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-30b88f17-516a-4d9c-9afb-0256dc133767 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:38:43.881322 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b88f17-516a-4d9c-9afb-0256dc133767 None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:38:43.893498 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd2a1a2-df30-4647-b9f7-97cabcda12e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:38:43.894443 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd2a1a2-df30-4647-b9f7-97cabcda12e2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:38:43.894795 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd2a1a2-df30-4647-b9f7-97cabcda12e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:38:43.895133 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd2a1a2-df30-4647-b9f7-97cabcda12e2 admin admin] PATH_INFO: `/v3/projects/2c93153688f646fca7018d9edfe5574b` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:38:43.896405 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cd2a1a2-df30-4647-b9f7-97cabcda12e2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2c93153688f646fca7018d9edfe5574b)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:38:43.898568 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cd2a1a2-df30-4647-b9f7-97cabcda12e2 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:38:43.952994 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-2cd2a1a2-df30-4647-b9f7-97cabcda12e2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2c93153688f646fca7018d9edfe5574b', 'request_id': 'req-2cd2a1a2-df30-4647-b9f7-97cabcda12e2'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:38:43.958950 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-2cd2a1a2-df30-4647-b9f7-97cabcda12e2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2c93153688f646fca7018d9edfe5574b', 'request_id': 'req-2cd2a1a2-df30-4647-b9f7-97cabcda12e2'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:38:43.965998 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-2cd2a1a2-df30-4647-b9f7-97cabcda12e2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2c93153688f646fca7018d9edfe5574b', 'request_id': 'req-2cd2a1a2-df30-4647-b9f7-97cabcda12e2'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:38:43.968524 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=93,keystone:DELETE=6,keystone:INSERT=1 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:38:43.985071 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-2cd2a1a2-df30-4647-b9f7-97cabcda12e2 admin admin] The token cache is being invalidate because project 2c93153688f646fca7018d9edfe5574b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80248) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 05:38:43.985480 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-2cd2a1a2-df30-4647-b9f7-97cabcda12e2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:38:43.986212 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-2cd2a1a2-df30-4647-b9f7-97cabcda12e2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:38:43.992220 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1108/2214] 10.222.0.33 () {58 vars in 1232 bytes} [Mon Jul 25 05:38:43 2022] DELETE /identity/v3/projects/2c93153688f646fca7018d9edfe5574b => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 05:38:43.993486 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f6f310-cb44-4705-87d8-490978e9f6d5 service neutron] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:38:44.004164 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f6f310-cb44-4705-87d8-490978e9f6d5 service neutron] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:38:44.049894 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:38:44.072159 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bbdd87b-15f9-4eef-a9bd-c1e4c78fa8ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:38:44.073072 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bbdd87b-15f9-4eef-a9bd-c1e4c78fa8ff admin admin] REQUEST_METHOD: `DELETE` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:38:44.073361 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bbdd87b-15f9-4eef-a9bd-c1e4c78fa8ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:38:44.073664 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bbdd87b-15f9-4eef-a9bd-c1e4c78fa8ff admin admin] PATH_INFO: `/v3/users/56965a3203ef4d3f8cff806207b1a806` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:38:44.074918 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bbdd87b-15f9-4eef-a9bd-c1e4c78fa8ff admin admin] RBAC: Authorizing `identity:delete_user(user_id=56965a3203ef4d3f8cff806207b1a806)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:38:44.077330 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bbdd87b-15f9-4eef-a9bd-c1e4c78fa8ff admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:38:44.126643 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-0bbdd87b-15f9-4eef-a9bd-c1e4c78fa8ff admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '56965a3203ef4d3f8cff806207b1a806'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:38:44.127379 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-0bbdd87b-15f9-4eef-a9bd-c1e4c78fa8ff admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '56965a3203ef4d3f8cff806207b1a806'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:38:44.134765 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-0bbdd87b-15f9-4eef-a9bd-c1e4c78fa8ff admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '56965a3203ef4d3f8cff806207b1a806'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:38:44.146465 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-0bbdd87b-15f9-4eef-a9bd-c1e4c78fa8ff admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '56965a3203ef4d3f8cff806207b1a806'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:38:44.147130 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-0bbdd87b-15f9-4eef-a9bd-c1e4c78fa8ff admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '56965a3203ef4d3f8cff806207b1a806'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:38:44.161261 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1107/2215] 10.222.0.33 () {58 vars in 1223 bytes} [Mon Jul 25 05:38:43 2022] DELETE /identity/v3/users/56965a3203ef4d3f8cff806207b1a806 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 05:38:44.440798 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd2a1a2-df30-4647-b9f7-97cabcda12e2 admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:38:44.442798 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-2cd2a1a2-df30-4647-b9f7-97cabcda12e2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:38:44.459643 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd2a1a2-df30-4647-b9f7-97cabcda12e2 admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:38:44.520316 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b005b84-3ebd-447f-ab62-ec030fe20135 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:38:44.521214 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b005b84-3ebd-447f-ab62-ec030fe20135 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:38:44.521499 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b005b84-3ebd-447f-ab62-ec030fe20135 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:38:44.521772 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b005b84-3ebd-447f-ab62-ec030fe20135 admin admin] PATH_INFO: `/v3/projects/6935284cd57940499a1e24401a9f2aa6` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:38:44.522958 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b005b84-3ebd-447f-ab62-ec030fe20135 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6935284cd57940499a1e24401a9f2aa6)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:38:44.524902 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b005b84-3ebd-447f-ab62-ec030fe20135 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:38:44.571996 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-9b005b84-3ebd-447f-ab62-ec030fe20135 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6935284cd57940499a1e24401a9f2aa6', 'request_id': 'req-9b005b84-3ebd-447f-ab62-ec030fe20135'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:38:44.577935 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-9b005b84-3ebd-447f-ab62-ec030fe20135 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6935284cd57940499a1e24401a9f2aa6', 'request_id': 'req-9b005b84-3ebd-447f-ab62-ec030fe20135'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:38:44.584212 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-9b005b84-3ebd-447f-ab62-ec030fe20135 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6935284cd57940499a1e24401a9f2aa6', 'request_id': 'req-9b005b84-3ebd-447f-ab62-ec030fe20135'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:38:44.601598 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-9b005b84-3ebd-447f-ab62-ec030fe20135 admin admin] The token cache is being invalidate because project 6935284cd57940499a1e24401a9f2aa6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80248) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 05:38:44.601941 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-9b005b84-3ebd-447f-ab62-ec030fe20135 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:38:44.602751 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-9b005b84-3ebd-447f-ab62-ec030fe20135 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:38:44.604499 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1109/2216] 10.222.0.33 () {58 vars in 1232 bytes} [Mon Jul 25 05:38:44 2022] DELETE /identity/v3/projects/6935284cd57940499a1e24401a9f2aa6 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 05:38:44.609212 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bbdd87b-15f9-4eef-a9bd-c1e4c78fa8ff admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:38:44.611196 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-0bbdd87b-15f9-4eef-a9bd-c1e4c78fa8ff admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:38:44.625793 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bbdd87b-15f9-4eef-a9bd-c1e4c78fa8ff admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:38:44.695028 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e06840-6751-4c57-b71f-be8c9e0f725a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:38:44.695914 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e06840-6751-4c57-b71f-be8c9e0f725a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:38:44.696228 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e06840-6751-4c57-b71f-be8c9e0f725a admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:38:44.696505 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e06840-6751-4c57-b71f-be8c9e0f725a admin admin] PATH_INFO: `/v3/users/b67b27be926e42f3a076518a19a5e3c2` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:38:44.697748 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95e06840-6751-4c57-b71f-be8c9e0f725a admin admin] RBAC: Authorizing `identity:delete_user(user_id=b67b27be926e42f3a076518a19a5e3c2)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:38:44.699665 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95e06840-6751-4c57-b71f-be8c9e0f725a admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:38:44.762976 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-95e06840-6751-4c57-b71f-be8c9e0f725a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b67b27be926e42f3a076518a19a5e3c2'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:38:44.763703 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-95e06840-6751-4c57-b71f-be8c9e0f725a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b67b27be926e42f3a076518a19a5e3c2'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:38:44.770993 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-95e06840-6751-4c57-b71f-be8c9e0f725a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b67b27be926e42f3a076518a19a5e3c2'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:38:44.775527 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=82,keystone:DELETE=17,keystone:INSERT=1 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:38:44.785863 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-95e06840-6751-4c57-b71f-be8c9e0f725a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b67b27be926e42f3a076518a19a5e3c2'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:38:44.786812 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-95e06840-6751-4c57-b71f-be8c9e0f725a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b67b27be926e42f3a076518a19a5e3c2'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:38:44.808453 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1108/2217] 10.222.0.33 () {58 vars in 1223 bytes} [Mon Jul 25 05:38:44 2022] DELETE /identity/v3/users/b67b27be926e42f3a076518a19a5e3c2 => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 05:38:45.269950 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b005b84-3ebd-447f-ab62-ec030fe20135 admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:38:45.271541 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-9b005b84-3ebd-447f-ab62-ec030fe20135 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:38:45.285729 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b005b84-3ebd-447f-ab62-ec030fe20135 admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:38:45.347975 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8165b191-d898-4cfa-8ed5-32fa71305e00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:38:45.348887 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8165b191-d898-4cfa-8ed5-32fa71305e00 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:38:45.349199 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8165b191-d898-4cfa-8ed5-32fa71305e00 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:38:45.349525 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8165b191-d898-4cfa-8ed5-32fa71305e00 admin admin] PATH_INFO: `/v3/projects/ace7b756233b490a8f52550824f77d30` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:38:45.350733 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8165b191-d898-4cfa-8ed5-32fa71305e00 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ace7b756233b490a8f52550824f77d30)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:38:45.352668 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8165b191-d898-4cfa-8ed5-32fa71305e00 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:38:45.396530 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:DELETE=9,keystone:SELECT=90,keystone:INSERT=1 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:38:45.417136 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-8165b191-d898-4cfa-8ed5-32fa71305e00 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ace7b756233b490a8f52550824f77d30', 'request_id': 'req-8165b191-d898-4cfa-8ed5-32fa71305e00'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:38:45.423177 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-8165b191-d898-4cfa-8ed5-32fa71305e00 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ace7b756233b490a8f52550824f77d30', 'request_id': 'req-8165b191-d898-4cfa-8ed5-32fa71305e00'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:38:45.433088 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-8165b191-d898-4cfa-8ed5-32fa71305e00 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ace7b756233b490a8f52550824f77d30', 'request_id': 'req-8165b191-d898-4cfa-8ed5-32fa71305e00'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:38:45.449029 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-8165b191-d898-4cfa-8ed5-32fa71305e00 admin admin] The token cache is being invalidate because project ace7b756233b490a8f52550824f77d30 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80248) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 05:38:45.449325 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-8165b191-d898-4cfa-8ed5-32fa71305e00 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:38:45.450524 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-8165b191-d898-4cfa-8ed5-32fa71305e00 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:38:45.452080 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1110/2218] 10.222.0.33 () {58 vars in 1232 bytes} [Mon Jul 25 05:38:45 2022] DELETE /identity/v3/projects/ace7b756233b490a8f52550824f77d30 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 25 05:38:49.573176 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e06840-6751-4c57-b71f-be8c9e0f725a admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:38:49.574623 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-95e06840-6751-4c57-b71f-be8c9e0f725a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:38:49.589272 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e06840-6751-4c57-b71f-be8c9e0f725a admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:38:49.691654 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-303a44c9-f06b-4fd2-866c-391d1069dbca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:38:49.692549 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303a44c9-f06b-4fd2-866c-391d1069dbca admin admin] REQUEST_METHOD: `DELETE` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:38:49.692863 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303a44c9-f06b-4fd2-866c-391d1069dbca admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:38:49.693160 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303a44c9-f06b-4fd2-866c-391d1069dbca admin admin] PATH_INFO: `/v3/users/109d2fe55a8748aea6df5b0efdf18eb6` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:38:49.694967 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-303a44c9-f06b-4fd2-866c-391d1069dbca admin admin] RBAC: Authorizing `identity:delete_user(user_id=109d2fe55a8748aea6df5b0efdf18eb6)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:38:49.696952 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-303a44c9-f06b-4fd2-866c-391d1069dbca admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:38:49.757389 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-303a44c9-f06b-4fd2-866c-391d1069dbca admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '109d2fe55a8748aea6df5b0efdf18eb6'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:38:49.758066 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-303a44c9-f06b-4fd2-866c-391d1069dbca admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '109d2fe55a8748aea6df5b0efdf18eb6'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:38:49.765510 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-303a44c9-f06b-4fd2-866c-391d1069dbca admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '109d2fe55a8748aea6df5b0efdf18eb6'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:38:49.778170 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-303a44c9-f06b-4fd2-866c-391d1069dbca admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '109d2fe55a8748aea6df5b0efdf18eb6'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:38:49.778915 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-303a44c9-f06b-4fd2-866c-391d1069dbca admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '109d2fe55a8748aea6df5b0efdf18eb6'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:38:49.792664 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1109/2219] 10.222.0.33 () {58 vars in 1223 bytes} [Mon Jul 25 05:38:49 2022] DELETE /identity/v3/users/109d2fe55a8748aea6df5b0efdf18eb6 => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 05:38:49.798407 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244daff9-72a6-43e7-a83d-56fb9a822742 None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:38:49.798967 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244daff9-72a6-43e7-a83d-56fb9a822742 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:38:49.798967 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244daff9-72a6-43e7-a83d-56fb9a822742 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:38:49.819702 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-244daff9-72a6-43e7-a83d-56fb9a822742 None None] MFA Rules not processed for user `1eae540242d94c10b0ad33960a855f08`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:38:49.821099 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-244daff9-72a6-43e7-a83d-56fb9a822742 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:38:49.891154 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1111/2220] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 25 05:38:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:38:49.901916 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-303a44c9-f06b-4fd2-866c-391d1069dbca admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:38:49.903276 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-303a44c9-f06b-4fd2-866c-391d1069dbca admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:38:49.976872 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-303a44c9-f06b-4fd2-866c-391d1069dbca admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:38:49.980600 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a24e9c5-0584-41ea-91cc-83b59326216a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b2931200f99e43d79a50a9f6e02738e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92afba266cce4d1bad4134c351e74be3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:38:49.981288 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a24e9c5-0584-41ea-91cc-83b59326216a service neutron] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:38:49.981574 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a24e9c5-0584-41ea-91cc-83b59326216a service neutron] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:38:49.981847 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a24e9c5-0584-41ea-91cc-83b59326216a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:38:49.993010 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a24e9c5-0584-41ea-91cc-83b59326216a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:38:49.994442 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a24e9c5-0584-41ea-91cc-83b59326216a service neutron] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:38:50.000018 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:DELETE=13,keystone:SELECT=85,keystone:INSERT=2 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:38:50.005994 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1110/2221] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jul 25 05:38:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 05:38:50.317593 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-244daff9-72a6-43e7-a83d-56fb9a822742 None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:38:50.319402 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-244daff9-72a6-43e7-a83d-56fb9a822742 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:38:50.339342 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-244daff9-72a6-43e7-a83d-56fb9a822742 None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:38:50.350209 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b4f470-e62a-4d9b-aede-3d88e59722e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:38:50.351205 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b4f470-e62a-4d9b-aede-3d88e59722e8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:38:50.351591 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b4f470-e62a-4d9b-aede-3d88e59722e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:38:50.352425 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b4f470-e62a-4d9b-aede-3d88e59722e8 admin admin] PATH_INFO: `/v3/projects/63dbaa5c92514af9bd6d74576a5b2147` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:38:50.354215 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37b4f470-e62a-4d9b-aede-3d88e59722e8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=63dbaa5c92514af9bd6d74576a5b2147)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:38:50.366444 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37b4f470-e62a-4d9b-aede-3d88e59722e8 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:38:50.439749 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-37b4f470-e62a-4d9b-aede-3d88e59722e8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '63dbaa5c92514af9bd6d74576a5b2147', 'request_id': 'req-37b4f470-e62a-4d9b-aede-3d88e59722e8'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:38:50.445921 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-37b4f470-e62a-4d9b-aede-3d88e59722e8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '63dbaa5c92514af9bd6d74576a5b2147', 'request_id': 'req-37b4f470-e62a-4d9b-aede-3d88e59722e8'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:38:50.454327 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-37b4f470-e62a-4d9b-aede-3d88e59722e8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '63dbaa5c92514af9bd6d74576a5b2147', 'request_id': 'req-37b4f470-e62a-4d9b-aede-3d88e59722e8'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:38:50.469720 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-37b4f470-e62a-4d9b-aede-3d88e59722e8 admin admin] The token cache is being invalidate because project 63dbaa5c92514af9bd6d74576a5b2147 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80248) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 05:38:50.470015 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-37b4f470-e62a-4d9b-aede-3d88e59722e8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:38:50.470685 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-37b4f470-e62a-4d9b-aede-3d88e59722e8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:38:50.472249 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1112/2222] 10.222.0.33 () {58 vars in 1232 bytes} [Mon Jul 25 05:38:50 2022] DELETE /identity/v3/projects/63dbaa5c92514af9bd6d74576a5b2147 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 05:38:50.476335 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a24e9c5-0584-41ea-91cc-83b59326216a service neutron] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:38:50.486674 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a24e9c5-0584-41ea-91cc-83b59326216a service neutron] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:38:50.548904 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7adce735-3711-4849-8f05-fe37b1ed923b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:38:50.549801 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7adce735-3711-4849-8f05-fe37b1ed923b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:38:50.550087 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7adce735-3711-4849-8f05-fe37b1ed923b admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:38:50.550370 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7adce735-3711-4849-8f05-fe37b1ed923b admin admin] PATH_INFO: `/v3/users/4a5c272f9428406eb7ea2f8d1cedce5a` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:38:50.551602 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7adce735-3711-4849-8f05-fe37b1ed923b admin admin] RBAC: Authorizing `identity:delete_user(user_id=4a5c272f9428406eb7ea2f8d1cedce5a)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:38:50.553684 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7adce735-3711-4849-8f05-fe37b1ed923b admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:38:50.612598 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-7adce735-3711-4849-8f05-fe37b1ed923b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4a5c272f9428406eb7ea2f8d1cedce5a'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:38:50.613352 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-7adce735-3711-4849-8f05-fe37b1ed923b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4a5c272f9428406eb7ea2f8d1cedce5a'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:38:50.625281 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-7adce735-3711-4849-8f05-fe37b1ed923b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4a5c272f9428406eb7ea2f8d1cedce5a'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:38:50.640807 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-7adce735-3711-4849-8f05-fe37b1ed923b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4a5c272f9428406eb7ea2f8d1cedce5a'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:38:50.641910 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-7adce735-3711-4849-8f05-fe37b1ed923b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4a5c272f9428406eb7ea2f8d1cedce5a'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:38:50.656834 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1111/2223] 10.222.0.33 () {58 vars in 1223 bytes} [Mon Jul 25 05:38:50 2022] DELETE /identity/v3/users/4a5c272f9428406eb7ea2f8d1cedce5a => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 05:38:51.047294 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b4f470-e62a-4d9b-aede-3d88e59722e8 admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:38:51.049806 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-37b4f470-e62a-4d9b-aede-3d88e59722e8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:38:51.076765 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b4f470-e62a-4d9b-aede-3d88e59722e8 admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:38:51.122405 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:DELETE=13,keystone:SELECT=85,keystone:INSERT=2 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:38:51.158709 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f1be45-55b4-4c34-ae9c-6fbe774da8af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:38:51.159755 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f1be45-55b4-4c34-ae9c-6fbe774da8af admin admin] REQUEST_METHOD: `DELETE` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:38:51.160045 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f1be45-55b4-4c34-ae9c-6fbe774da8af admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:38:51.160323 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f1be45-55b4-4c34-ae9c-6fbe774da8af admin admin] PATH_INFO: `/v3/projects/a91f41ec2e2746e9a4ecf04cf091bc2a` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:38:51.162063 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84f1be45-55b4-4c34-ae9c-6fbe774da8af admin admin] RBAC: Authorizing `identity:delete_project(project_id=a91f41ec2e2746e9a4ecf04cf091bc2a)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:38:51.164084 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84f1be45-55b4-4c34-ae9c-6fbe774da8af admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:38:51.231850 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-84f1be45-55b4-4c34-ae9c-6fbe774da8af admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a91f41ec2e2746e9a4ecf04cf091bc2a', 'request_id': 'req-84f1be45-55b4-4c34-ae9c-6fbe774da8af'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:38:51.243775 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-84f1be45-55b4-4c34-ae9c-6fbe774da8af admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a91f41ec2e2746e9a4ecf04cf091bc2a', 'request_id': 'req-84f1be45-55b4-4c34-ae9c-6fbe774da8af'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:38:51.253416 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-84f1be45-55b4-4c34-ae9c-6fbe774da8af admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a91f41ec2e2746e9a4ecf04cf091bc2a', 'request_id': 'req-84f1be45-55b4-4c34-ae9c-6fbe774da8af'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:38:51.282782 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-84f1be45-55b4-4c34-ae9c-6fbe774da8af admin admin] The token cache is being invalidate because project a91f41ec2e2746e9a4ecf04cf091bc2a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80248) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 05:38:51.283036 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-84f1be45-55b4-4c34-ae9c-6fbe774da8af admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:38:51.284001 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-84f1be45-55b4-4c34-ae9c-6fbe774da8af admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:38:51.285691 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1113/2224] 10.222.0.33 () {58 vars in 1232 bytes} [Mon Jul 25 05:38:51 2022] DELETE /identity/v3/projects/a91f41ec2e2746e9a4ecf04cf091bc2a => generated 0 bytes in 239 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 05:38:51.291220 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e2b6b3-bcef-4d6a-a012-1b5e67d2ea5f None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:38:51.291538 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e2b6b3-bcef-4d6a-a012-1b5e67d2ea5f None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:38:51.292069 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e2b6b3-bcef-4d6a-a012-1b5e67d2ea5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:38:51.312422 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-f3e2b6b3-bcef-4d6a-a012-1b5e67d2ea5f None None] MFA Rules not processed for user `1eae540242d94c10b0ad33960a855f08`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:38:51.313729 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-f3e2b6b3-bcef-4d6a-a012-1b5e67d2ea5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:38:51.384595 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1112/2225] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 25 05:38:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:38:51.393688 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c76621-d5cb-400d-9f6c-840ec6bf7f94 None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:38:51.395291 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c76621-d5cb-400d-9f6c-840ec6bf7f94 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:38:51.395736 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c76621-d5cb-400d-9f6c-840ec6bf7f94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:38:51.415272 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-b1c76621-d5cb-400d-9f6c-840ec6bf7f94 None None] MFA Rules not processed for user `1eae540242d94c10b0ad33960a855f08`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:38:51.416441 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-b1c76621-d5cb-400d-9f6c-840ec6bf7f94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:38:51.425138 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1114/2226] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 25 05:38:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:38:51.430704 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e2b6b3-bcef-4d6a-a012-1b5e67d2ea5f None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:38:51.442283 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e2b6b3-bcef-4d6a-a012-1b5e67d2ea5f None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:38:51.447532 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-074c7c7f-9d57-4847-ae8b-ed1f2b6f89a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:38:51.448288 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074c7c7f-9d57-4847-ae8b-ed1f2b6f89a9 admin admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:38:51.448669 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074c7c7f-9d57-4847-ae8b-ed1f2b6f89a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:38:51.449052 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074c7c7f-9d57-4847-ae8b-ed1f2b6f89a9 admin admin] PATH_INFO: `/v3/domains` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:38:51.450070 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-074c7c7f-9d57-4847-ae8b-ed1f2b6f89a9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:38:51.450456 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-074c7c7f-9d57-4847-ae8b-ed1f2b6f89a9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:38:51.452322 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-074c7c7f-9d57-4847-ae8b-ed1f2b6f89a9 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:38:51.471702 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1113/2227] 10.222.0.33 () {58 vars in 1152 bytes} [Mon Jul 25 05:38:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:38:51.473887 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c76621-d5cb-400d-9f6c-840ec6bf7f94 None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:38:51.487996 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c76621-d5cb-400d-9f6c-840ec6bf7f94 None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:38:51.494336 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0709859d-02da-412b-9f74-d398a3184018 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:38:51.495084 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0709859d-02da-412b-9f74-d398a3184018 admin admin] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:38:51.495373 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0709859d-02da-412b-9f74-d398a3184018 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:38:51.495651 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0709859d-02da-412b-9f74-d398a3184018 admin admin] PATH_INFO: `/v3/projects` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:38:51.496445 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0709859d-02da-412b-9f74-d398a3184018 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:38:51.499031 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0709859d-02da-412b-9f74-d398a3184018 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:38:51.531170 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1115/2228] 10.222.0.33 () {60 vars in 1152 bytes} [Mon Jul 25 05:38:51 2022] POST /identity/v3/projects => generated 383 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 25 05:38:51.532499 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-074c7c7f-9d57-4847-ae8b-ed1f2b6f89a9 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:38:51.542569 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-074c7c7f-9d57-4847-ae8b-ed1f2b6f89a9 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:38:51.547027 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:38:51.610396 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ee1734-f1af-49ee-810a-244d3d104f19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:38:51.611086 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ee1734-f1af-49ee-810a-244d3d104f19 admin admin] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:38:51.611380 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ee1734-f1af-49ee-810a-244d3d104f19 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:38:51.611665 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ee1734-f1af-49ee-810a-244d3d104f19 admin admin] PATH_INFO: `/v3/users` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:38:51.612517 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51ee1734-f1af-49ee-810a-244d3d104f19 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:38:51.614602 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51ee1734-f1af-49ee-810a-244d3d104f19 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:38:51.636707 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1114/2229] 10.222.0.33 () {60 vars in 1143 bytes} [Mon Jul 25 05:38:51 2022] POST /identity/v3/users => generated 292 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 25 05:38:51.641102 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0709859d-02da-412b-9f74-d398a3184018 admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:38:51.652649 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0709859d-02da-412b-9f74-d398a3184018 admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:38:51.658701 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86885d58-1654-4343-a5c9-2f127b60cb4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:38:51.659133 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86885d58-1654-4343-a5c9-2f127b60cb4c admin admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:38:51.659509 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86885d58-1654-4343-a5c9-2f127b60cb4c admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:38:51.659847 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86885d58-1654-4343-a5c9-2f127b60cb4c admin admin] PATH_INFO: `/v3/roles` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:38:51.660654 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86885d58-1654-4343-a5c9-2f127b60cb4c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:38:51.662355 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86885d58-1654-4343-a5c9-2f127b60cb4c admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:38:51.676868 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1116/2230] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jul 25 05:38:51 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 05:38:51.677790 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ee1734-f1af-49ee-810a-244d3d104f19 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:38:51.689639 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ee1734-f1af-49ee-810a-244d3d104f19 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:38:51.694300 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c969a1-d0ff-4eae-b8ee-f6a075b8deb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:38:51.695049 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c969a1-d0ff-4eae-b8ee-f6a075b8deb7 admin admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:38:51.695334 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c969a1-d0ff-4eae-b8ee-f6a075b8deb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:38:51.695611 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c969a1-d0ff-4eae-b8ee-f6a075b8deb7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:38:51.696345 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29c969a1-d0ff-4eae-b8ee-f6a075b8deb7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:38:51.697996 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29c969a1-d0ff-4eae-b8ee-f6a075b8deb7 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:38:51.711147 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1115/2231] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jul 25 05:38:51 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 25 05:38:51.715250 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86885d58-1654-4343-a5c9-2f127b60cb4c admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:38:51.726304 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86885d58-1654-4343-a5c9-2f127b60cb4c admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:38:51.731632 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec6fa8b-9c56-4564-a874-0b6189c4b4a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:38:51.733152 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec6fa8b-9c56-4564-a874-0b6189c4b4a6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:38:51.733438 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec6fa8b-9c56-4564-a874-0b6189c4b4a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:38:51.733722 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec6fa8b-9c56-4564-a874-0b6189c4b4a6 admin admin] PATH_INFO: `/v3/projects/53ab1f11e0b64b82a39b148902f34fed/users/0ec43aafcbf2454cb3213cb39730e688/roles/5074e74e35f54d4b9bb415c34999a82a` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:38:51.750450 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ec6fa8b-9c56-4564-a874-0b6189c4b4a6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=53ab1f11e0b64b82a39b148902f34fed, user_id=0ec43aafcbf2454cb3213cb39730e688, role_id=5074e74e35f54d4b9bb415c34999a82a)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:38:51.753402 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ec6fa8b-9c56-4564-a874-0b6189c4b4a6 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:38:51.767888 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1117/2232] 10.222.0.33 () {60 vars in 1482 bytes} [Mon Jul 25 05:38:51 2022] PUT /identity/v3/projects/53ab1f11e0b64b82a39b148902f34fed/users/0ec43aafcbf2454cb3213cb39730e688/roles/5074e74e35f54d4b9bb415c34999a82a => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 25 05:38:51.778858 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d367d8-e544-4c15-829e-af8c5eed16ea None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:38:51.779290 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d367d8-e544-4c15-829e-af8c5eed16ea None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:38:51.779608 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d367d8-e544-4c15-829e-af8c5eed16ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:38:51.808981 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-80d367d8-e544-4c15-829e-af8c5eed16ea None None] MFA Rules not processed for user `0ec43aafcbf2454cb3213cb39730e688`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:38:51.810327 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-80d367d8-e544-4c15-829e-af8c5eed16ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:38:51.878952 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1116/2233] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 25 05:38:51 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:38:51.883510 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec6fa8b-9c56-4564-a874-0b6189c4b4a6 admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:38:51.946895 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec6fa8b-9c56-4564-a874-0b6189c4b4a6 admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:38:51.951636 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e320c80-b8a2-4e48-8d47-c82641373bd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:38:51.952156 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e320c80-b8a2-4e48-8d47-c82641373bd5 admin admin] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:38:51.952446 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e320c80-b8a2-4e48-8d47-c82641373bd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:38:51.952729 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e320c80-b8a2-4e48-8d47-c82641373bd5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:38:51.953465 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e320c80-b8a2-4e48-8d47-c82641373bd5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:38:51.955425 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e320c80-b8a2-4e48-8d47-c82641373bd5 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:38:51.964062 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:38:51.988585 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1118/2234] 10.222.0.33 () {60 vars in 1152 bytes} [Mon Jul 25 05:38:51 2022] POST /identity/v3/projects => generated 387 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 05:38:51.990019 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d367d8-e544-4c15-829e-af8c5eed16ea None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:38:52.002544 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d367d8-e544-4c15-829e-af8c5eed16ea None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:38:52.064377 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-290e2af1-5595-4d87-b156-210e3a0abcd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:38:52.065225 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290e2af1-5595-4d87-b156-210e3a0abcd9 admin admin] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:38:52.065610 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290e2af1-5595-4d87-b156-210e3a0abcd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:38:52.065987 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290e2af1-5595-4d87-b156-210e3a0abcd9 admin admin] PATH_INFO: `/v3/users` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:38:52.067011 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-290e2af1-5595-4d87-b156-210e3a0abcd9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:38:52.069303 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-290e2af1-5595-4d87-b156-210e3a0abcd9 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:38:52.099032 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1117/2235] 10.222.0.33 () {60 vars in 1143 bytes} [Mon Jul 25 05:38:51 2022] POST /identity/v3/users => generated 300 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 05:38:52.103134 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e320c80-b8a2-4e48-8d47-c82641373bd5 admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:38:52.125106 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e320c80-b8a2-4e48-8d47-c82641373bd5 admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:38:52.130407 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d1059e8-4828-4a68-86f0-bbeecb640e1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:38:52.131350 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1059e8-4828-4a68-86f0-bbeecb640e1a admin admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:38:52.131704 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1059e8-4828-4a68-86f0-bbeecb640e1a admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:38:52.132051 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1059e8-4828-4a68-86f0-bbeecb640e1a admin admin] PATH_INFO: `/v3/roles` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:38:52.132885 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d1059e8-4828-4a68-86f0-bbeecb640e1a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:38:52.134854 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d1059e8-4828-4a68-86f0-bbeecb640e1a admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:38:52.148808 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1119/2236] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jul 25 05:38:52 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 25 05:38:52.151799 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-290e2af1-5595-4d87-b156-210e3a0abcd9 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:38:52.158822 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:38:52.163848 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-290e2af1-5595-4d87-b156-210e3a0abcd9 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:38:52.170833 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d7357d6-6dfe-4d01-9aeb-5942476e8c9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:38:52.171744 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7357d6-6dfe-4d01-9aeb-5942476e8c9f admin admin] REQUEST_METHOD: `PUT` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:38:52.172027 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7357d6-6dfe-4d01-9aeb-5942476e8c9f admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:38:52.172305 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7357d6-6dfe-4d01-9aeb-5942476e8c9f admin admin] PATH_INFO: `/v3/projects/a5b85550a3614743864373ca398f68a3/users/b35eb04a235d49b8bc72a0e6905ac067/roles/39e5c666285a42fe8fc2c3afca6260d1` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:38:52.187390 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d7357d6-6dfe-4d01-9aeb-5942476e8c9f admin admin] RBAC: Authorizing `identity:create_grant(project_id=a5b85550a3614743864373ca398f68a3, user_id=b35eb04a235d49b8bc72a0e6905ac067, role_id=39e5c666285a42fe8fc2c3afca6260d1)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:38:52.190084 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d7357d6-6dfe-4d01-9aeb-5942476e8c9f admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:38:52.203008 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1118/2237] 10.222.0.33 () {60 vars in 1482 bytes} [Mon Jul 25 05:38:52 2022] PUT /identity/v3/projects/a5b85550a3614743864373ca398f68a3/users/b35eb04a235d49b8bc72a0e6905ac067/roles/39e5c666285a42fe8fc2c3afca6260d1 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 05:38:52.206848 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d1059e8-4828-4a68-86f0-bbeecb640e1a admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:38:52.218698 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d1059e8-4828-4a68-86f0-bbeecb640e1a admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:38:52.275940 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d480351-5ca2-4542-914a-9e83f6003e3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:38:52.276615 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d480351-5ca2-4542-914a-9e83f6003e3e admin admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:38:52.276916 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d480351-5ca2-4542-914a-9e83f6003e3e admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:38:52.277195 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d480351-5ca2-4542-914a-9e83f6003e3e admin admin] PATH_INFO: `/v3/roles` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:38:52.277923 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d480351-5ca2-4542-914a-9e83f6003e3e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:38:52.280178 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d480351-5ca2-4542-914a-9e83f6003e3e admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:38:52.289821 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1120/2238] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jul 25 05:38:52 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 05:38:52.293985 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d7357d6-6dfe-4d01-9aeb-5942476e8c9f admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:38:52.304695 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d7357d6-6dfe-4d01-9aeb-5942476e8c9f admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:38:52.309724 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b1726a6-d72b-49c5-a8cc-5faa4de54f9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:38:52.310635 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1726a6-d72b-49c5-a8cc-5faa4de54f9a admin admin] REQUEST_METHOD: `PUT` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:38:52.310921 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1726a6-d72b-49c5-a8cc-5faa4de54f9a admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:38:52.311201 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1726a6-d72b-49c5-a8cc-5faa4de54f9a admin admin] PATH_INFO: `/v3/projects/a5b85550a3614743864373ca398f68a3/users/b35eb04a235d49b8bc72a0e6905ac067/roles/39e5c666285a42fe8fc2c3afca6260d1` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:38:52.313240 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b1726a6-d72b-49c5-a8cc-5faa4de54f9a admin admin] RBAC: Authorizing `identity:create_grant(project_id=a5b85550a3614743864373ca398f68a3, user_id=b35eb04a235d49b8bc72a0e6905ac067, role_id=39e5c666285a42fe8fc2c3afca6260d1)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:38:52.315865 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b1726a6-d72b-49c5-a8cc-5faa4de54f9a admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:38:52.326041 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1119/2239] 10.222.0.33 () {60 vars in 1482 bytes} [Mon Jul 25 05:38:52 2022] PUT /identity/v3/projects/a5b85550a3614743864373ca398f68a3/users/b35eb04a235d49b8bc72a0e6905ac067/roles/39e5c666285a42fe8fc2c3afca6260d1 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 05:38:52.336367 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c704809c-6d77-40d7-b4b8-a13464aa309b None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:38:52.336643 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c704809c-6d77-40d7-b4b8-a13464aa309b None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:38:52.336969 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c704809c-6d77-40d7-b4b8-a13464aa309b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:38:52.364401 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-c704809c-6d77-40d7-b4b8-a13464aa309b None None] MFA Rules not processed for user `b35eb04a235d49b8bc72a0e6905ac067`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:38:52.365627 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-c704809c-6d77-40d7-b4b8-a13464aa309b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:38:52.431189 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1121/2240] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 25 05:38:52 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:38:52.442291 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b1726a6-d72b-49c5-a8cc-5faa4de54f9a admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:38:52.443997 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-1b1726a6-d72b-49c5-a8cc-5faa4de54f9a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:38:52.504293 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b1726a6-d72b-49c5-a8cc-5faa4de54f9a admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:38:52.508577 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aecba8f3-45f3-4bb2-89d2-45fc83876d66 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db57b478c25643dbaf936f90f6e17d0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92afba266cce4d1bad4134c351e74be3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:38:52.509260 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aecba8f3-45f3-4bb2-89d2-45fc83876d66 service cinder] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:38:52.509548 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aecba8f3-45f3-4bb2-89d2-45fc83876d66 service cinder] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:38:52.509828 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aecba8f3-45f3-4bb2-89d2-45fc83876d66 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:38:52.519789 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aecba8f3-45f3-4bb2-89d2-45fc83876d66 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:38:52.521256 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aecba8f3-45f3-4bb2-89d2-45fc83876d66 service cinder] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:38:52.531965 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1120/2241] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jul 25 05:38:52 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 05:38:53.174120 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c704809c-6d77-40d7-b4b8-a13464aa309b None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:38:53.266287 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c704809c-6d77-40d7-b4b8-a13464aa309b None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:38:53.271112 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c0d8f6-e6f6-47ab-8995-6ac1f4c4e164 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:38:53.271986 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c0d8f6-e6f6-47ab-8995-6ac1f4c4e164 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:38:53.272430 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c0d8f6-e6f6-47ab-8995-6ac1f4c4e164 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:38:53.272710 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c0d8f6-e6f6-47ab-8995-6ac1f4c4e164 admin admin] PATH_INFO: `/v3/users/0ec43aafcbf2454cb3213cb39730e688` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:38:53.273958 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74c0d8f6-e6f6-47ab-8995-6ac1f4c4e164 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0ec43aafcbf2454cb3213cb39730e688)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:38:53.275908 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74c0d8f6-e6f6-47ab-8995-6ac1f4c4e164 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:38:53.277764 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:38:53.341677 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aecba8f3-45f3-4bb2-89d2-45fc83876d66 service cinder] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:38:53.342880 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-74c0d8f6-e6f6-47ab-8995-6ac1f4c4e164 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0ec43aafcbf2454cb3213cb39730e688'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:38:53.344992 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-aecba8f3-45f3-4bb2-89d2-45fc83876d66 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:38:53.357817 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-74c0d8f6-e6f6-47ab-8995-6ac1f4c4e164 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0ec43aafcbf2454cb3213cb39730e688'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:38:53.358729 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-74c0d8f6-e6f6-47ab-8995-6ac1f4c4e164 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0ec43aafcbf2454cb3213cb39730e688'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:38:53.360164 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aecba8f3-45f3-4bb2-89d2-45fc83876d66 service cinder] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:38:53.364919 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04777b6f-5c40-4dfa-b4c9-6925aeaa3e6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:38:53.366686 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04777b6f-5c40-4dfa-b4c9-6925aeaa3e6a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:38:53.367308 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04777b6f-5c40-4dfa-b4c9-6925aeaa3e6a admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:38:53.367652 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04777b6f-5c40-4dfa-b4c9-6925aeaa3e6a admin admin] PATH_INFO: `/v3/users/30e5711bc03a4ca888084a599f217a0f` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:38:53.368723 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-74c0d8f6-e6f6-47ab-8995-6ac1f4c4e164 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0ec43aafcbf2454cb3213cb39730e688'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:38:53.369199 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04777b6f-5c40-4dfa-b4c9-6925aeaa3e6a admin admin] RBAC: Authorizing `identity:delete_user(user_id=30e5711bc03a4ca888084a599f217a0f)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:38:53.369493 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-74c0d8f6-e6f6-47ab-8995-6ac1f4c4e164 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0ec43aafcbf2454cb3213cb39730e688'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:38:53.371863 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04777b6f-5c40-4dfa-b4c9-6925aeaa3e6a admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:38:53.393192 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1122/2242] 10.222.0.33 () {58 vars in 1223 bytes} [Mon Jul 25 05:38:53 2022] DELETE /identity/v3/users/0ec43aafcbf2454cb3213cb39730e688 => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 05:38:53.398720 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f23b686-e3c9-4ace-b5f8-b21b6d30b233 None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:38:53.399076 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f23b686-e3c9-4ace-b5f8-b21b6d30b233 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:38:53.399426 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f23b686-e3c9-4ace-b5f8-b21b6d30b233 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:38:53.419007 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-0f23b686-e3c9-4ace-b5f8-b21b6d30b233 None None] MFA Rules not processed for user `1eae540242d94c10b0ad33960a855f08`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:38:53.420670 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-0f23b686-e3c9-4ace-b5f8-b21b6d30b233 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:38:53.440446 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-04777b6f-5c40-4dfa-b4c9-6925aeaa3e6a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '30e5711bc03a4ca888084a599f217a0f'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:38:53.441291 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-04777b6f-5c40-4dfa-b4c9-6925aeaa3e6a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '30e5711bc03a4ca888084a599f217a0f'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:38:53.449910 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-04777b6f-5c40-4dfa-b4c9-6925aeaa3e6a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '30e5711bc03a4ca888084a599f217a0f'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:38:53.466187 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-04777b6f-5c40-4dfa-b4c9-6925aeaa3e6a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '30e5711bc03a4ca888084a599f217a0f'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:38:53.467536 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-04777b6f-5c40-4dfa-b4c9-6925aeaa3e6a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '30e5711bc03a4ca888084a599f217a0f'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:38:53.480917 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1121/2243] 10.222.0.33 () {58 vars in 1223 bytes} [Mon Jul 25 05:38:53 2022] DELETE /identity/v3/users/30e5711bc03a4ca888084a599f217a0f => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 05:38:53.485394 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b94975d-c2b4-40cd-bf6a-ffb082f164af None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:38:53.485603 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b94975d-c2b4-40cd-bf6a-ffb082f164af None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:38:53.485887 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b94975d-c2b4-40cd-bf6a-ffb082f164af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:38:53.498456 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1123/2244] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 25 05:38:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 05:38:53.504105 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-7b94975d-c2b4-40cd-bf6a-ffb082f164af None None] MFA Rules not processed for user `1eae540242d94c10b0ad33960a855f08`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:38:53.506691 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-7b94975d-c2b4-40cd-bf6a-ffb082f164af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:38:53.507301 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f23b686-e3c9-4ace-b5f8-b21b6d30b233 None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:38:53.509295 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-0f23b686-e3c9-4ace-b5f8-b21b6d30b233 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:38:53.551707 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=87,keystone:INSERT=3,keystone:DELETE=10 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:38:53.580327 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1122/2245] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 25 05:38:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 05:38:53.584769 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f23b686-e3c9-4ace-b5f8-b21b6d30b233 None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:38:53.592070 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b94975d-c2b4-40cd-bf6a-ffb082f164af None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:38:53.625375 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:38:53.647302 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6406b28-51f0-47d8-90a0-2def61cba98e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b2931200f99e43d79a50a9f6e02738e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92afba266cce4d1bad4134c351e74be3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:38:53.648081 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6406b28-51f0-47d8-90a0-2def61cba98e service neutron] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:38:53.648478 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6406b28-51f0-47d8-90a0-2def61cba98e service neutron] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:38:53.648832 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6406b28-51f0-47d8-90a0-2def61cba98e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:38:53.661708 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6406b28-51f0-47d8-90a0-2def61cba98e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:38:53.663339 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b94975d-c2b4-40cd-bf6a-ffb082f164af None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:38:53.663729 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6406b28-51f0-47d8-90a0-2def61cba98e service neutron] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:38:53.667745 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed27c5a-a1b8-49d2-95b2-f1bcefa5c104 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b2931200f99e43d79a50a9f6e02738e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92afba266cce4d1bad4134c351e74be3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:38:53.668411 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed27c5a-a1b8-49d2-95b2-f1bcefa5c104 service neutron] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:38:53.668702 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed27c5a-a1b8-49d2-95b2-f1bcefa5c104 service neutron] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:38:53.669003 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed27c5a-a1b8-49d2-95b2-f1bcefa5c104 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:38:53.676373 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1124/2246] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jul 25 05:38:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 170 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 05:38:53.680090 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ed27c5a-a1b8-49d2-95b2-f1bcefa5c104 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:38:53.681627 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ed27c5a-a1b8-49d2-95b2-f1bcefa5c104 service neutron] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:38:53.693479 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1123/2247] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jul 25 05:38:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 05:38:54.195530 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6406b28-51f0-47d8-90a0-2def61cba98e service neutron] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:38:54.197394 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-b6406b28-51f0-47d8-90a0-2def61cba98e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:38:54.206510 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed27c5a-a1b8-49d2-95b2-f1bcefa5c104 service neutron] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:38:54.208923 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-8ed27c5a-a1b8-49d2-95b2-f1bcefa5c104 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:38:54.216351 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6406b28-51f0-47d8-90a0-2def61cba98e service neutron] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:38:54.227497 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-214bd7af-2677-4cd1-a9b1-aec9dbbd5889 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:38:54.228384 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214bd7af-2677-4cd1-a9b1-aec9dbbd5889 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:38:54.228668 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214bd7af-2677-4cd1-a9b1-aec9dbbd5889 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:38:54.229040 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214bd7af-2677-4cd1-a9b1-aec9dbbd5889 admin admin] PATH_INFO: `/v3/projects/35d2f0aee1784bdba7727f70fd41c28e` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:38:54.230304 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-214bd7af-2677-4cd1-a9b1-aec9dbbd5889 admin admin] RBAC: Authorizing `identity:delete_project(project_id=35d2f0aee1784bdba7727f70fd41c28e)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:38:54.232257 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-214bd7af-2677-4cd1-a9b1-aec9dbbd5889 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:38:54.256195 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed27c5a-a1b8-49d2-95b2-f1bcefa5c104 service neutron] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:38:54.261256 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f913566b-ad11-482c-9ce2-ceb366206b37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:38:54.262196 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f913566b-ad11-482c-9ce2-ceb366206b37 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:38:54.262564 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f913566b-ad11-482c-9ce2-ceb366206b37 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:38:54.262908 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f913566b-ad11-482c-9ce2-ceb366206b37 admin admin] PATH_INFO: `/v3/projects/53ab1f11e0b64b82a39b148902f34fed` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:38:54.268117 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f913566b-ad11-482c-9ce2-ceb366206b37 admin admin] RBAC: Authorizing `identity:delete_project(project_id=53ab1f11e0b64b82a39b148902f34fed)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:38:54.276592 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f913566b-ad11-482c-9ce2-ceb366206b37 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:38:54.294310 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-214bd7af-2677-4cd1-a9b1-aec9dbbd5889 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '35d2f0aee1784bdba7727f70fd41c28e', 'request_id': 'req-214bd7af-2677-4cd1-a9b1-aec9dbbd5889'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:38:54.300920 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-214bd7af-2677-4cd1-a9b1-aec9dbbd5889 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '35d2f0aee1784bdba7727f70fd41c28e', 'request_id': 'req-214bd7af-2677-4cd1-a9b1-aec9dbbd5889'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:38:54.308156 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-214bd7af-2677-4cd1-a9b1-aec9dbbd5889 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '35d2f0aee1784bdba7727f70fd41c28e', 'request_id': 'req-214bd7af-2677-4cd1-a9b1-aec9dbbd5889'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:38:54.324481 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-214bd7af-2677-4cd1-a9b1-aec9dbbd5889 admin admin] The token cache is being invalidate because project 35d2f0aee1784bdba7727f70fd41c28e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80248) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 05:38:54.325110 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-214bd7af-2677-4cd1-a9b1-aec9dbbd5889 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:38:54.326497 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-214bd7af-2677-4cd1-a9b1-aec9dbbd5889 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:38:54.329289 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1125/2248] 10.222.0.33 () {58 vars in 1232 bytes} [Mon Jul 25 05:38:54 2022] DELETE /identity/v3/projects/35d2f0aee1784bdba7727f70fd41c28e => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 05:38:54.335135 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c60b887-7a95-42fe-9709-3339ac9af273 None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:38:54.335662 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c60b887-7a95-42fe-9709-3339ac9af273 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:38:54.335936 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c60b887-7a95-42fe-9709-3339ac9af273 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:38:54.345086 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-f913566b-ad11-482c-9ce2-ceb366206b37 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '53ab1f11e0b64b82a39b148902f34fed', 'request_id': 'req-f913566b-ad11-482c-9ce2-ceb366206b37'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:38:54.351439 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-f913566b-ad11-482c-9ce2-ceb366206b37 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '53ab1f11e0b64b82a39b148902f34fed', 'request_id': 'req-f913566b-ad11-482c-9ce2-ceb366206b37'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:38:54.357875 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-f913566b-ad11-482c-9ce2-ceb366206b37 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '53ab1f11e0b64b82a39b148902f34fed', 'request_id': 'req-f913566b-ad11-482c-9ce2-ceb366206b37'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:38:54.358771 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-9c60b887-7a95-42fe-9709-3339ac9af273 None None] MFA Rules not processed for user `1eae540242d94c10b0ad33960a855f08`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:38:54.361148 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-9c60b887-7a95-42fe-9709-3339ac9af273 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:38:54.376959 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-f913566b-ad11-482c-9ce2-ceb366206b37 admin admin] The token cache is being invalidate because project 53ab1f11e0b64b82a39b148902f34fed was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80249) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 05:38:54.377704 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-f913566b-ad11-482c-9ce2-ceb366206b37 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:38:54.379316 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-f913566b-ad11-482c-9ce2-ceb366206b37 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:38:54.381607 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1124/2249] 10.222.0.33 () {58 vars in 1232 bytes} [Mon Jul 25 05:38:54 2022] DELETE /identity/v3/projects/53ab1f11e0b64b82a39b148902f34fed => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 05:38:54.385531 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f913566b-ad11-482c-9ce2-ceb366206b37 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:38:54.386916 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-f913566b-ad11-482c-9ce2-ceb366206b37 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:38:54.443946 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1126/2250] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 25 05:38:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:38:54.452803 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c345aca-4420-4949-bd60-a0e0eda822cd None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:38:54.453876 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c345aca-4420-4949-bd60-a0e0eda822cd None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:38:54.454165 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c345aca-4420-4949-bd60-a0e0eda822cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:38:54.473825 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f913566b-ad11-482c-9ce2-ceb366206b37 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:38:54.478982 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cecdaf0-2a4d-4f6a-b58f-e8f2e11d8b5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:38:54.479259 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-7c345aca-4420-4949-bd60-a0e0eda822cd None None] MFA Rules not processed for user `1eae540242d94c10b0ad33960a855f08`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:38:54.479954 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cecdaf0-2a4d-4f6a-b58f-e8f2e11d8b5c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:38:54.480241 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cecdaf0-2a4d-4f6a-b58f-e8f2e11d8b5c admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:38:54.480558 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-7c345aca-4420-4949-bd60-a0e0eda822cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:38:54.480749 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cecdaf0-2a4d-4f6a-b58f-e8f2e11d8b5c admin admin] PATH_INFO: `/v3/users/b35eb04a235d49b8bc72a0e6905ac067` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:38:54.481986 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cecdaf0-2a4d-4f6a-b58f-e8f2e11d8b5c admin admin] RBAC: Authorizing `identity:delete_user(user_id=b35eb04a235d49b8bc72a0e6905ac067)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:38:54.484263 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cecdaf0-2a4d-4f6a-b58f-e8f2e11d8b5c admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:38:54.489348 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1127/2251] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 25 05:38:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:38:54.493617 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c345aca-4420-4949-bd60-a0e0eda822cd None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:38:54.505403 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:38:54.507070 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c345aca-4420-4949-bd60-a0e0eda822cd None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:38:54.513970 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d18e1d75-1802-458b-952e-046237207c26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:38:54.514679 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18e1d75-1802-458b-952e-046237207c26 admin admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:38:54.515050 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18e1d75-1802-458b-952e-046237207c26 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:38:54.515222 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18e1d75-1802-458b-952e-046237207c26 admin admin] PATH_INFO: `/v3/domains` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:38:54.515966 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d18e1d75-1802-458b-952e-046237207c26 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:38:54.516277 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d18e1d75-1802-458b-952e-046237207c26 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:38:54.517951 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d18e1d75-1802-458b-952e-046237207c26 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:38:54.534144 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1128/2252] 10.222.0.33 () {58 vars in 1152 bytes} [Mon Jul 25 05:38:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 05:38:54.538388 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d18e1d75-1802-458b-952e-046237207c26 admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:38:54.556004 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-5cecdaf0-2a4d-4f6a-b58f-e8f2e11d8b5c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b35eb04a235d49b8bc72a0e6905ac067'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:38:54.556882 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-5cecdaf0-2a4d-4f6a-b58f-e8f2e11d8b5c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b35eb04a235d49b8bc72a0e6905ac067'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:38:54.557761 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d18e1d75-1802-458b-952e-046237207c26 admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:38:54.566256 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a5774dd-8f67-4db4-9750-855fcd0d4db9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:38:54.567372 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5774dd-8f67-4db4-9750-855fcd0d4db9 admin admin] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:38:54.567651 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-5cecdaf0-2a4d-4f6a-b58f-e8f2e11d8b5c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b35eb04a235d49b8bc72a0e6905ac067'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:38:54.567850 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5774dd-8f67-4db4-9750-855fcd0d4db9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:38:54.568229 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5774dd-8f67-4db4-9750-855fcd0d4db9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:38:54.569343 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a5774dd-8f67-4db4-9750-855fcd0d4db9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:38:54.572707 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a5774dd-8f67-4db4-9750-855fcd0d4db9 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:38:54.591394 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-5cecdaf0-2a4d-4f6a-b58f-e8f2e11d8b5c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b35eb04a235d49b8bc72a0e6905ac067'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:38:54.593223 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-5cecdaf0-2a4d-4f6a-b58f-e8f2e11d8b5c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b35eb04a235d49b8bc72a0e6905ac067'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:38:54.618030 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1129/2253] 10.222.0.33 () {60 vars in 1152 bytes} [Mon Jul 25 05:38:54 2022] POST /identity/v3/projects => generated 371 bytes in 77 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 25 05:38:54.619612 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a5774dd-8f67-4db4-9750-855fcd0d4db9 admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:38:54.622164 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-6a5774dd-8f67-4db4-9750-855fcd0d4db9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:38:54.635862 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1125/2254] 10.222.0.33 () {58 vars in 1223 bytes} [Mon Jul 25 05:38:54 2022] DELETE /identity/v3/users/b35eb04a235d49b8bc72a0e6905ac067 => generated 0 bytes in 251 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 25 05:38:54.662085 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:38:54.727958 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a5774dd-8f67-4db4-9750-855fcd0d4db9 admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:38:54.793533 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b1e0c3-2672-44d1-932b-56d0084b355c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:38:54.794209 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b1e0c3-2672-44d1-932b-56d0084b355c admin admin] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:38:54.794500 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b1e0c3-2672-44d1-932b-56d0084b355c admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:38:54.795433 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b1e0c3-2672-44d1-932b-56d0084b355c admin admin] PATH_INFO: `/v3/users` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:38:54.796007 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51b1e0c3-2672-44d1-932b-56d0084b355c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:38:54.798025 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51b1e0c3-2672-44d1-932b-56d0084b355c admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:38:54.824227 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1130/2255] 10.222.0.33 () {60 vars in 1143 bytes} [Mon Jul 25 05:38:54 2022] POST /identity/v3/users => generated 286 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 05:38:54.825430 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cecdaf0-2a4d-4f6a-b58f-e8f2e11d8b5c admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:38:54.839956 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cecdaf0-2a4d-4f6a-b58f-e8f2e11d8b5c admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:38:54.845702 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af4035b5-c72b-4eb0-9a04-f89db5d3aee6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:38:54.846452 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4035b5-c72b-4eb0-9a04-f89db5d3aee6 admin admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:38:54.846807 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4035b5-c72b-4eb0-9a04-f89db5d3aee6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:38:54.847155 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4035b5-c72b-4eb0-9a04-f89db5d3aee6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:38:54.847958 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af4035b5-c72b-4eb0-9a04-f89db5d3aee6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:38:54.849891 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af4035b5-c72b-4eb0-9a04-f89db5d3aee6 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:38:54.860930 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1126/2256] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jul 25 05:38:54 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 05:38:54.865037 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b1e0c3-2672-44d1-932b-56d0084b355c admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:38:54.875821 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b1e0c3-2672-44d1-932b-56d0084b355c admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:38:54.880492 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7159cfb5-7b84-4a51-b1d6-f51edd97e21c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:38:54.881359 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7159cfb5-7b84-4a51-b1d6-f51edd97e21c admin admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:38:54.881714 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7159cfb5-7b84-4a51-b1d6-f51edd97e21c admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:38:54.882054 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7159cfb5-7b84-4a51-b1d6-f51edd97e21c admin admin] PATH_INFO: `/v3/roles` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:38:54.882849 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7159cfb5-7b84-4a51-b1d6-f51edd97e21c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:38:54.884497 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7159cfb5-7b84-4a51-b1d6-f51edd97e21c admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:38:54.895664 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1131/2257] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jul 25 05:38:54 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 05:38:54.900146 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af4035b5-c72b-4eb0-9a04-f89db5d3aee6 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:38:54.910365 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af4035b5-c72b-4eb0-9a04-f89db5d3aee6 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:38:54.915179 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7286a31b-755d-49c3-9d11-0004892c885c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:38:54.916060 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7286a31b-755d-49c3-9d11-0004892c885c admin admin] REQUEST_METHOD: `PUT` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:38:54.916344 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7286a31b-755d-49c3-9d11-0004892c885c admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:38:54.916623 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7286a31b-755d-49c3-9d11-0004892c885c admin admin] PATH_INFO: `/v3/projects/9f4184b7029846d08de47f3b73346243/users/c0118c891dc74e30b9493b303e52db12/roles/5074e74e35f54d4b9bb415c34999a82a` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:38:54.933527 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7286a31b-755d-49c3-9d11-0004892c885c admin admin] RBAC: Authorizing `identity:create_grant(project_id=9f4184b7029846d08de47f3b73346243, user_id=c0118c891dc74e30b9493b303e52db12, role_id=5074e74e35f54d4b9bb415c34999a82a)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:38:54.936704 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7286a31b-755d-49c3-9d11-0004892c885c admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:38:54.950919 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1127/2258] 10.222.0.33 () {60 vars in 1482 bytes} [Mon Jul 25 05:38:54 2022] PUT /identity/v3/projects/9f4184b7029846d08de47f3b73346243/users/c0118c891dc74e30b9493b303e52db12/roles/5074e74e35f54d4b9bb415c34999a82a => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 05:38:54.954067 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7159cfb5-7b84-4a51-b1d6-f51edd97e21c admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:38:54.955920 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-7159cfb5-7b84-4a51-b1d6-f51edd97e21c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:38:54.962774 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71b452d-9f6f-4d1c-8184-9233d380b156 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:38:54.963050 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71b452d-9f6f-4d1c-8184-9233d380b156 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:38:54.963534 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71b452d-9f6f-4d1c-8184-9233d380b156 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:38:54.975388 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7159cfb5-7b84-4a51-b1d6-f51edd97e21c admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:38:54.995542 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-f71b452d-9f6f-4d1c-8184-9233d380b156 None None] MFA Rules not processed for user `c0118c891dc74e30b9493b303e52db12`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:38:54.996799 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-f71b452d-9f6f-4d1c-8184-9233d380b156 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:38:55.039414 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b066fb0-fb95-4d97-9216-31c73d7bcca8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:38:55.040180 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b066fb0-fb95-4d97-9216-31c73d7bcca8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:38:55.040487 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b066fb0-fb95-4d97-9216-31c73d7bcca8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:38:55.040763 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b066fb0-fb95-4d97-9216-31c73d7bcca8 admin admin] PATH_INFO: `/v3/projects/a5b85550a3614743864373ca398f68a3` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:38:55.041992 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b066fb0-fb95-4d97-9216-31c73d7bcca8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a5b85550a3614743864373ca398f68a3)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:38:55.044201 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b066fb0-fb95-4d97-9216-31c73d7bcca8 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:38:55.070325 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1128/2259] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 25 05:38:54 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 05:38:55.081870 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f71b452d-9f6f-4d1c-8184-9233d380b156 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:38:55.083584 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-f71b452d-9f6f-4d1c-8184-9233d380b156 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:38:55.094514 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=93,keystone:INSERT=3,keystone:DELETE=4 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:38:55.095840 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-3b066fb0-fb95-4d97-9216-31c73d7bcca8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a5b85550a3614743864373ca398f68a3', 'request_id': 'req-3b066fb0-fb95-4d97-9216-31c73d7bcca8'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:38:55.103383 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-3b066fb0-fb95-4d97-9216-31c73d7bcca8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a5b85550a3614743864373ca398f68a3', 'request_id': 'req-3b066fb0-fb95-4d97-9216-31c73d7bcca8'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:38:55.116879 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-3b066fb0-fb95-4d97-9216-31c73d7bcca8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a5b85550a3614743864373ca398f68a3', 'request_id': 'req-3b066fb0-fb95-4d97-9216-31c73d7bcca8'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:38:55.133666 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-3b066fb0-fb95-4d97-9216-31c73d7bcca8 admin admin] The token cache is being invalidate because project a5b85550a3614743864373ca398f68a3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80248) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 05:38:55.134134 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-3b066fb0-fb95-4d97-9216-31c73d7bcca8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:38:55.134846 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-3b066fb0-fb95-4d97-9216-31c73d7bcca8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:38:55.136616 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1132/2260] 10.222.0.33 () {58 vars in 1232 bytes} [Mon Jul 25 05:38:54 2022] DELETE /identity/v3/projects/a5b85550a3614743864373ca398f68a3 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 05:38:55.141560 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:DELETE=10,keystone:SELECT=88,keystone:INSERT=2 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:38:55.141695 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1562c5f3-9abf-4501-aaa5-a96a6b14bc15 None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:38:55.141880 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1562c5f3-9abf-4501-aaa5-a96a6b14bc15 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:38:55.142305 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1562c5f3-9abf-4501-aaa5-a96a6b14bc15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:38:55.157289 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f71b452d-9f6f-4d1c-8184-9233d380b156 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:38:55.162583 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-1562c5f3-9abf-4501-aaa5-a96a6b14bc15 None None] MFA Rules not processed for user `1eae540242d94c10b0ad33960a855f08`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:38:55.163953 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-1562c5f3-9abf-4501-aaa5-a96a6b14bc15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:38:55.226800 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b80e165d-2582-40d0-b477-1c7dae253de0 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de99e44bdf9c4030adbe9c0efa3831cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92afba266cce4d1bad4134c351e74be3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:38:55.227694 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80e165d-2582-40d0-b477-1c7dae253de0 service glance] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:38:55.228083 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80e165d-2582-40d0-b477-1c7dae253de0 service glance] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:38:55.228475 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80e165d-2582-40d0-b477-1c7dae253de0 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:38:55.230532 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-b80e165d-2582-40d0-b477-1c7dae253de0 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:38:55.235079 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1133/2261] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 25 05:38:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 05:38:55.247162 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b63e70f-5f68-4050-a36f-d7268c39c53d None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:38:55.247162 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b63e70f-5f68-4050-a36f-d7268c39c53d None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:38:55.247162 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b63e70f-5f68-4050-a36f-d7268c39c53d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:38:55.266956 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-5b63e70f-5f68-4050-a36f-d7268c39c53d None None] MFA Rules not processed for user `1eae540242d94c10b0ad33960a855f08`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:38:55.268187 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-5b63e70f-5f68-4050-a36f-d7268c39c53d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:38:55.280815 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1134/2262] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 25 05:38:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 05:38:55.283588 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b63e70f-5f68-4050-a36f-d7268c39c53d None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:38:55.293437 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b63e70f-5f68-4050-a36f-d7268c39c53d None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:38:55.298621 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7815a95a-9567-4a9b-83ca-c3bb02e276bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:38:55.299399 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7815a95a-9567-4a9b-83ca-c3bb02e276bc admin admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:38:55.299689 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7815a95a-9567-4a9b-83ca-c3bb02e276bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:38:55.299968 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7815a95a-9567-4a9b-83ca-c3bb02e276bc admin admin] PATH_INFO: `/v3/domains` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:38:55.301023 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7815a95a-9567-4a9b-83ca-c3bb02e276bc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:38:55.301538 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7815a95a-9567-4a9b-83ca-c3bb02e276bc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:38:55.303152 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7815a95a-9567-4a9b-83ca-c3bb02e276bc admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:38:55.320675 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1135/2263] 10.222.0.33 () {58 vars in 1152 bytes} [Mon Jul 25 05:38:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 05:38:55.321387 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b80e165d-2582-40d0-b477-1c7dae253de0 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:38:55.324267 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b80e165d-2582-40d0-b477-1c7dae253de0 service glance] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:38:55.324517 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7815a95a-9567-4a9b-83ca-c3bb02e276bc admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:38:55.336761 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1129/2264] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jul 25 05:38:55 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 255 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 25 05:38:55.337245 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7815a95a-9567-4a9b-83ca-c3bb02e276bc admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:38:55.342817 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac0b397-a7e1-455f-a552-1bd029523f38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:38:55.343476 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac0b397-a7e1-455f-a552-1bd029523f38 admin admin] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:38:55.343773 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac0b397-a7e1-455f-a552-1bd029523f38 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:38:55.344034 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac0b397-a7e1-455f-a552-1bd029523f38 admin admin] PATH_INFO: `/v3/projects` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:38:55.344956 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fac0b397-a7e1-455f-a552-1bd029523f38 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:38:55.348204 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fac0b397-a7e1-455f-a552-1bd029523f38 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:38:55.389917 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1136/2265] 10.222.0.33 () {60 vars in 1152 bytes} [Mon Jul 25 05:38:55 2022] POST /identity/v3/projects => generated 375 bytes in 66 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 25 05:38:55.395769 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b80e165d-2582-40d0-b477-1c7dae253de0 service glance] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:38:55.415226 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b80e165d-2582-40d0-b477-1c7dae253de0 service glance] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:38:55.477166 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb8301d-bb73-46d0-849f-08a526e3963b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:38:55.477864 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb8301d-bb73-46d0-849f-08a526e3963b admin admin] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:38:55.478146 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb8301d-bb73-46d0-849f-08a526e3963b admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:38:55.478419 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb8301d-bb73-46d0-849f-08a526e3963b admin admin] PATH_INFO: `/v3/users` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:38:55.479135 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fb8301d-bb73-46d0-849f-08a526e3963b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:38:55.481106 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fb8301d-bb73-46d0-849f-08a526e3963b admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:38:55.510522 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1130/2266] 10.222.0.33 () {60 vars in 1143 bytes} [Mon Jul 25 05:38:55 2022] POST /identity/v3/users => generated 288 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 05:38:55.511447 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac0b397-a7e1-455f-a552-1bd029523f38 admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:38:55.522396 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac0b397-a7e1-455f-a552-1bd029523f38 admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:38:55.527125 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f252f27f-ebb6-445e-877f-8d7994ca9d4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:38:55.527793 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f252f27f-ebb6-445e-877f-8d7994ca9d4d admin admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:38:55.528082 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f252f27f-ebb6-445e-877f-8d7994ca9d4d admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:38:55.528364 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f252f27f-ebb6-445e-877f-8d7994ca9d4d admin admin] PATH_INFO: `/v3/roles` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:38:55.529150 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f252f27f-ebb6-445e-877f-8d7994ca9d4d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:38:55.531378 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f252f27f-ebb6-445e-877f-8d7994ca9d4d admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:38:55.542514 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1137/2267] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jul 25 05:38:55 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 25 05:38:55.547380 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb8301d-bb73-46d0-849f-08a526e3963b admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:38:55.557672 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb8301d-bb73-46d0-849f-08a526e3963b admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:38:55.563398 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-724f4559-5ba4-455a-a57e-32ed4a1def90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:38:55.564162 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724f4559-5ba4-455a-a57e-32ed4a1def90 admin admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:38:55.564449 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724f4559-5ba4-455a-a57e-32ed4a1def90 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:38:55.564733 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724f4559-5ba4-455a-a57e-32ed4a1def90 admin admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:38:55.565495 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-724f4559-5ba4-455a-a57e-32ed4a1def90 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:38:55.567129 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-724f4559-5ba4-455a-a57e-32ed4a1def90 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:38:55.581108 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1131/2268] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jul 25 05:38:55 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 05:38:55.582620 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f252f27f-ebb6-445e-877f-8d7994ca9d4d admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:38:55.592628 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f252f27f-ebb6-445e-877f-8d7994ca9d4d admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:38:55.600950 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edefa446-28ae-4ba8-a582-74b5dfa7fb3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:38:55.602172 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edefa446-28ae-4ba8-a582-74b5dfa7fb3c admin admin] REQUEST_METHOD: `PUT` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:38:55.602431 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edefa446-28ae-4ba8-a582-74b5dfa7fb3c admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:38:55.602721 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edefa446-28ae-4ba8-a582-74b5dfa7fb3c admin admin] PATH_INFO: `/v3/projects/c6d22108c30d4a36a470c687d96d290c/users/e90cc02ac55f4aada79d82017973632a/roles/5074e74e35f54d4b9bb415c34999a82a` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:38:55.618940 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edefa446-28ae-4ba8-a582-74b5dfa7fb3c admin admin] RBAC: Authorizing `identity:create_grant(project_id=c6d22108c30d4a36a470c687d96d290c, user_id=e90cc02ac55f4aada79d82017973632a, role_id=5074e74e35f54d4b9bb415c34999a82a)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:38:55.621846 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edefa446-28ae-4ba8-a582-74b5dfa7fb3c admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:38:55.640545 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1138/2269] 10.222.0.33 () {60 vars in 1482 bytes} [Mon Jul 25 05:38:55 2022] PUT /identity/v3/projects/c6d22108c30d4a36a470c687d96d290c/users/e90cc02ac55f4aada79d82017973632a/roles/5074e74e35f54d4b9bb415c34999a82a => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 05:38:55.651402 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3d89a4-2a5f-434a-a167-e91d46fbf7e4 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:38:55.651718 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3d89a4-2a5f-434a-a167-e91d46fbf7e4 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:38:55.652045 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3d89a4-2a5f-434a-a167-e91d46fbf7e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:38:55.686228 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-8c3d89a4-2a5f-434a-a167-e91d46fbf7e4 None None] MFA Rules not processed for user `e90cc02ac55f4aada79d82017973632a`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:38:55.687683 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-8c3d89a4-2a5f-434a-a167-e91d46fbf7e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:38:55.708755 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:38:55.770504 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1132/2270] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 25 05:38:55 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:38:55.775022 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edefa446-28ae-4ba8-a582-74b5dfa7fb3c admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:38:55.862994 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edefa446-28ae-4ba8-a582-74b5dfa7fb3c admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:38:55.868086 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa8e268c-8d1e-4dd2-a659-f17a9f824172 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:38:55.868749 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8e268c-8d1e-4dd2-a659-f17a9f824172 admin admin] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:38:55.869071 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8e268c-8d1e-4dd2-a659-f17a9f824172 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:38:55.869349 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8e268c-8d1e-4dd2-a659-f17a9f824172 admin admin] PATH_INFO: `/v3/projects` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:38:55.870042 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa8e268c-8d1e-4dd2-a659-f17a9f824172 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:38:55.871932 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa8e268c-8d1e-4dd2-a659-f17a9f824172 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:38:55.875825 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=94,keystone:DELETE=3,keystone:INSERT=3 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:38:55.902232 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1139/2271] 10.222.0.33 () {60 vars in 1152 bytes} [Mon Jul 25 05:38:55 2022] POST /identity/v3/projects => generated 371 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 05:38:55.907219 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c3d89a4-2a5f-434a-a167-e91d46fbf7e4 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:38:55.917961 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c3d89a4-2a5f-434a-a167-e91d46fbf7e4 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:38:55.975951 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30526af2-c691-4e64-93e5-5bca134863a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:38:55.976624 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30526af2-c691-4e64-93e5-5bca134863a6 admin admin] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:38:55.976945 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30526af2-c691-4e64-93e5-5bca134863a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:38:55.977228 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30526af2-c691-4e64-93e5-5bca134863a6 admin admin] PATH_INFO: `/v3/users` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:38:55.977932 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30526af2-c691-4e64-93e5-5bca134863a6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:38:55.979848 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30526af2-c691-4e64-93e5-5bca134863a6 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:38:56.001556 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1133/2272] 10.222.0.33 () {60 vars in 1143 bytes} [Mon Jul 25 05:38:55 2022] POST /identity/v3/users => generated 292 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 05:38:56.005552 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa8e268c-8d1e-4dd2-a659-f17a9f824172 admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:38:56.016253 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa8e268c-8d1e-4dd2-a659-f17a9f824172 admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:38:56.020825 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c011bd0d-4363-4438-8d49-e2fbf26a508b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:38:56.021484 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c011bd0d-4363-4438-8d49-e2fbf26a508b admin admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:38:56.021766 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c011bd0d-4363-4438-8d49-e2fbf26a508b admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:38:56.022043 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c011bd0d-4363-4438-8d49-e2fbf26a508b admin admin] PATH_INFO: `/v3/roles` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:38:56.022775 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c011bd0d-4363-4438-8d49-e2fbf26a508b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:38:56.024439 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c011bd0d-4363-4438-8d49-e2fbf26a508b admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:38:56.035453 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1140/2273] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jul 25 05:38:56 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 05:38:56.039785 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30526af2-c691-4e64-93e5-5bca134863a6 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:38:56.050159 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30526af2-c691-4e64-93e5-5bca134863a6 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:38:56.054756 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b197568-dae6-45fc-a0b6-07be57d5e5e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:38:56.055637 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b197568-dae6-45fc-a0b6-07be57d5e5e0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:38:56.055923 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b197568-dae6-45fc-a0b6-07be57d5e5e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:38:56.056201 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b197568-dae6-45fc-a0b6-07be57d5e5e0 admin admin] PATH_INFO: `/v3/projects/cf0ecccdccb146ca9ac25814bab43a75/users/e3ce2823dbb54dce93c29b08e77100a1/roles/39e5c666285a42fe8fc2c3afca6260d1` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:38:56.071259 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b197568-dae6-45fc-a0b6-07be57d5e5e0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cf0ecccdccb146ca9ac25814bab43a75, user_id=e3ce2823dbb54dce93c29b08e77100a1, role_id=39e5c666285a42fe8fc2c3afca6260d1)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:38:56.073988 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b197568-dae6-45fc-a0b6-07be57d5e5e0 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:38:56.087438 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1134/2274] 10.222.0.33 () {60 vars in 1482 bytes} [Mon Jul 25 05:38:56 2022] PUT /identity/v3/projects/cf0ecccdccb146ca9ac25814bab43a75/users/e3ce2823dbb54dce93c29b08e77100a1/roles/39e5c666285a42fe8fc2c3afca6260d1 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 05:38:56.091320 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c011bd0d-4363-4438-8d49-e2fbf26a508b admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:38:56.101728 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c011bd0d-4363-4438-8d49-e2fbf26a508b admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:38:56.159883 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a9eaa2d-c881-4c49-8abc-709fc93f0b7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:38:56.160547 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9eaa2d-c881-4c49-8abc-709fc93f0b7e admin admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:38:56.160864 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9eaa2d-c881-4c49-8abc-709fc93f0b7e admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:38:56.161146 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9eaa2d-c881-4c49-8abc-709fc93f0b7e admin admin] PATH_INFO: `/v3/roles` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:38:56.161873 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a9eaa2d-c881-4c49-8abc-709fc93f0b7e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:38:56.163465 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a9eaa2d-c881-4c49-8abc-709fc93f0b7e admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:38:56.174351 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1141/2275] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jul 25 05:38:56 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 05:38:56.178422 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b197568-dae6-45fc-a0b6-07be57d5e5e0 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:38:56.188615 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b197568-dae6-45fc-a0b6-07be57d5e5e0 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:38:56.193177 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8535138-48ad-4c5d-a681-53aa36c62887 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:38:56.194046 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8535138-48ad-4c5d-a681-53aa36c62887 admin admin] REQUEST_METHOD: `PUT` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:38:56.194326 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8535138-48ad-4c5d-a681-53aa36c62887 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:38:56.194600 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8535138-48ad-4c5d-a681-53aa36c62887 admin admin] PATH_INFO: `/v3/projects/cf0ecccdccb146ca9ac25814bab43a75/users/e3ce2823dbb54dce93c29b08e77100a1/roles/39e5c666285a42fe8fc2c3afca6260d1` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:38:56.196561 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8535138-48ad-4c5d-a681-53aa36c62887 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cf0ecccdccb146ca9ac25814bab43a75, user_id=e3ce2823dbb54dce93c29b08e77100a1, role_id=39e5c666285a42fe8fc2c3afca6260d1)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:38:56.199212 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8535138-48ad-4c5d-a681-53aa36c62887 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:38:56.210983 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1135/2276] 10.222.0.33 () {60 vars in 1482 bytes} [Mon Jul 25 05:38:56 2022] PUT /identity/v3/projects/cf0ecccdccb146ca9ac25814bab43a75/users/e3ce2823dbb54dce93c29b08e77100a1/roles/39e5c666285a42fe8fc2c3afca6260d1 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 05:38:56.222263 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39502da-a97a-4473-8cb3-8408200df69f None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:38:56.222557 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39502da-a97a-4473-8cb3-8408200df69f None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:38:56.222841 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39502da-a97a-4473-8cb3-8408200df69f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:38:56.249760 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-a39502da-a97a-4473-8cb3-8408200df69f None None] MFA Rules not processed for user `e3ce2823dbb54dce93c29b08e77100a1`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:38:56.250972 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-a39502da-a97a-4473-8cb3-8408200df69f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:38:56.317614 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1142/2277] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 25 05:38:56 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:38:56.329256 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8535138-48ad-4c5d-a681-53aa36c62887 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:38:56.330926 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-b8535138-48ad-4c5d-a681-53aa36c62887 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:38:56.391429 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8535138-48ad-4c5d-a681-53aa36c62887 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:38:56.395070 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af0e9628-c0c6-43d1-b64e-8a2e50528f45 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db57b478c25643dbaf936f90f6e17d0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92afba266cce4d1bad4134c351e74be3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:38:56.395710 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0e9628-c0c6-43d1-b64e-8a2e50528f45 service cinder] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:38:56.395997 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0e9628-c0c6-43d1-b64e-8a2e50528f45 service cinder] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:38:56.396547 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0e9628-c0c6-43d1-b64e-8a2e50528f45 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:38:56.424694 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:38:56.469523 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af0e9628-c0c6-43d1-b64e-8a2e50528f45 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:38:56.471004 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af0e9628-c0c6-43d1-b64e-8a2e50528f45 service cinder] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:38:56.480585 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1136/2278] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jul 25 05:38:56 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 152 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 05:39:01.167661 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a39502da-a97a-4473-8cb3-8408200df69f None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:39:01.239446 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:39:01.247273 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a39502da-a97a-4473-8cb3-8408200df69f None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:39:01.253186 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-136a3dbd-678c-4d4f-9f59-316030460ab5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:39:01.254141 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136a3dbd-678c-4d4f-9f59-316030460ab5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:39:01.254683 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136a3dbd-678c-4d4f-9f59-316030460ab5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:39:01.255045 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136a3dbd-678c-4d4f-9f59-316030460ab5 admin admin] PATH_INFO: `/v3/users/e90cc02ac55f4aada79d82017973632a` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:39:01.256940 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-136a3dbd-678c-4d4f-9f59-316030460ab5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e90cc02ac55f4aada79d82017973632a)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:39:01.259590 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-136a3dbd-678c-4d4f-9f59-316030460ab5 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:39:01.324395 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-136a3dbd-678c-4d4f-9f59-316030460ab5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e90cc02ac55f4aada79d82017973632a'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:39:01.339163 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-136a3dbd-678c-4d4f-9f59-316030460ab5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e90cc02ac55f4aada79d82017973632a'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:39:01.340275 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-136a3dbd-678c-4d4f-9f59-316030460ab5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e90cc02ac55f4aada79d82017973632a'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:39:01.351035 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-136a3dbd-678c-4d4f-9f59-316030460ab5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e90cc02ac55f4aada79d82017973632a'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:39:01.351789 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-136a3dbd-678c-4d4f-9f59-316030460ab5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e90cc02ac55f4aada79d82017973632a'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:39:01.375519 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1143/2279] 10.222.0.33 () {58 vars in 1223 bytes} [Mon Jul 25 05:39:01 2022] DELETE /identity/v3/users/e90cc02ac55f4aada79d82017973632a => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 25 05:39:01.380475 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052aaa7a-4b58-4a42-a9d5-64a470a7499e None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:39:01.380925 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052aaa7a-4b58-4a42-a9d5-64a470a7499e None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:39:01.381084 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052aaa7a-4b58-4a42-a9d5-64a470a7499e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:39:01.401150 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-052aaa7a-4b58-4a42-a9d5-64a470a7499e None None] MFA Rules not processed for user `1eae540242d94c10b0ad33960a855f08`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:39:01.402387 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-052aaa7a-4b58-4a42-a9d5-64a470a7499e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:39:01.472036 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1137/2280] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 25 05:39:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:39:01.483944 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-136a3dbd-678c-4d4f-9f59-316030460ab5 admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:39:01.485549 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-136a3dbd-678c-4d4f-9f59-316030460ab5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:39:01.544287 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-136a3dbd-678c-4d4f-9f59-316030460ab5 admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:39:01.548705 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45fb4a09-5fef-4b46-a32e-5d416dea52b6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b2931200f99e43d79a50a9f6e02738e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92afba266cce4d1bad4134c351e74be3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:39:01.549663 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45fb4a09-5fef-4b46-a32e-5d416dea52b6 service neutron] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:39:01.549982 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45fb4a09-5fef-4b46-a32e-5d416dea52b6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:39:01.550266 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45fb4a09-5fef-4b46-a32e-5d416dea52b6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:39:01.560233 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45fb4a09-5fef-4b46-a32e-5d416dea52b6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:39:01.561781 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45fb4a09-5fef-4b46-a32e-5d416dea52b6 service neutron] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:39:01.572603 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1144/2281] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jul 25 05:39:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 05:39:01.855659 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-052aaa7a-4b58-4a42-a9d5-64a470a7499e None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:39:01.857143 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-052aaa7a-4b58-4a42-a9d5-64a470a7499e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:39:01.875740 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-052aaa7a-4b58-4a42-a9d5-64a470a7499e None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:39:01.880425 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe145e26-118c-4539-96a6-4f55af66a14e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:39:01.881322 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe145e26-118c-4539-96a6-4f55af66a14e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:39:01.881616 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe145e26-118c-4539-96a6-4f55af66a14e admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:39:01.881882 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe145e26-118c-4539-96a6-4f55af66a14e admin admin] PATH_INFO: `/v3/projects/c6d22108c30d4a36a470c687d96d290c` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:39:01.883105 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe145e26-118c-4539-96a6-4f55af66a14e admin admin] RBAC: Authorizing `identity:delete_project(project_id=c6d22108c30d4a36a470c687d96d290c)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:39:01.885018 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe145e26-118c-4539-96a6-4f55af66a14e admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:39:01.931250 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-fe145e26-118c-4539-96a6-4f55af66a14e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c6d22108c30d4a36a470c687d96d290c', 'request_id': 'req-fe145e26-118c-4539-96a6-4f55af66a14e'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:39:01.937646 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-fe145e26-118c-4539-96a6-4f55af66a14e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c6d22108c30d4a36a470c687d96d290c', 'request_id': 'req-fe145e26-118c-4539-96a6-4f55af66a14e'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:39:01.944485 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-fe145e26-118c-4539-96a6-4f55af66a14e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c6d22108c30d4a36a470c687d96d290c', 'request_id': 'req-fe145e26-118c-4539-96a6-4f55af66a14e'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:39:01.960544 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-fe145e26-118c-4539-96a6-4f55af66a14e admin admin] The token cache is being invalidate because project c6d22108c30d4a36a470c687d96d290c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80249) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 05:39:01.960869 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-fe145e26-118c-4539-96a6-4f55af66a14e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:39:01.961552 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-fe145e26-118c-4539-96a6-4f55af66a14e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:39:01.963116 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1138/2282] 10.222.0.33 () {58 vars in 1232 bytes} [Mon Jul 25 05:39:01 2022] DELETE /identity/v3/projects/c6d22108c30d4a36a470c687d96d290c => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 05:39:01.967846 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45fb4a09-5fef-4b46-a32e-5d416dea52b6 service neutron] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:39:01.980038 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45fb4a09-5fef-4b46-a32e-5d416dea52b6 service neutron] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:39:02.044611 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab9e0d44-0e86-4f97-a3fb-523e694cc519 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:39:02.046098 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9e0d44-0e86-4f97-a3fb-523e694cc519 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:39:02.046438 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9e0d44-0e86-4f97-a3fb-523e694cc519 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:39:02.046758 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9e0d44-0e86-4f97-a3fb-523e694cc519 admin admin] PATH_INFO: `/v3/users/e3ce2823dbb54dce93c29b08e77100a1` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:39:02.048109 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab9e0d44-0e86-4f97-a3fb-523e694cc519 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e3ce2823dbb54dce93c29b08e77100a1)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:39:02.050082 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab9e0d44-0e86-4f97-a3fb-523e694cc519 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:39:02.072463 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:39:02.099768 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-ab9e0d44-0e86-4f97-a3fb-523e694cc519 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e3ce2823dbb54dce93c29b08e77100a1'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:39:02.113535 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-ab9e0d44-0e86-4f97-a3fb-523e694cc519 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e3ce2823dbb54dce93c29b08e77100a1'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:39:02.114373 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-ab9e0d44-0e86-4f97-a3fb-523e694cc519 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e3ce2823dbb54dce93c29b08e77100a1'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:39:02.122886 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-ab9e0d44-0e86-4f97-a3fb-523e694cc519 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e3ce2823dbb54dce93c29b08e77100a1'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:39:02.123718 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-ab9e0d44-0e86-4f97-a3fb-523e694cc519 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e3ce2823dbb54dce93c29b08e77100a1'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:39:02.137932 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1145/2283] 10.222.0.33 () {58 vars in 1223 bytes} [Mon Jul 25 05:39:01 2022] DELETE /identity/v3/users/e3ce2823dbb54dce93c29b08e77100a1 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 25 05:39:02.434346 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe145e26-118c-4539-96a6-4f55af66a14e admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:39:02.436063 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-fe145e26-118c-4539-96a6-4f55af66a14e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:39:02.451256 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe145e26-118c-4539-96a6-4f55af66a14e admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:39:02.507957 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:39:02.521208 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d9adb7d-7635-41ae-9dbd-5cb1267511d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:39:02.522152 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9adb7d-7635-41ae-9dbd-5cb1267511d5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:39:02.522565 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9adb7d-7635-41ae-9dbd-5cb1267511d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:39:02.522853 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9adb7d-7635-41ae-9dbd-5cb1267511d5 admin admin] PATH_INFO: `/v3/projects/cf0ecccdccb146ca9ac25814bab43a75` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:39:02.524077 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d9adb7d-7635-41ae-9dbd-5cb1267511d5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cf0ecccdccb146ca9ac25814bab43a75)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:39:02.526336 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d9adb7d-7635-41ae-9dbd-5cb1267511d5 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:39:02.581667 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-3d9adb7d-7635-41ae-9dbd-5cb1267511d5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cf0ecccdccb146ca9ac25814bab43a75', 'request_id': 'req-3d9adb7d-7635-41ae-9dbd-5cb1267511d5'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:39:02.587932 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-3d9adb7d-7635-41ae-9dbd-5cb1267511d5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cf0ecccdccb146ca9ac25814bab43a75', 'request_id': 'req-3d9adb7d-7635-41ae-9dbd-5cb1267511d5'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:39:02.594872 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-3d9adb7d-7635-41ae-9dbd-5cb1267511d5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cf0ecccdccb146ca9ac25814bab43a75', 'request_id': 'req-3d9adb7d-7635-41ae-9dbd-5cb1267511d5'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:39:02.607559 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-3d9adb7d-7635-41ae-9dbd-5cb1267511d5 admin admin] The token cache is being invalidate because project cf0ecccdccb146ca9ac25814bab43a75 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80249) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 05:39:02.607945 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-3d9adb7d-7635-41ae-9dbd-5cb1267511d5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:39:02.608698 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-3d9adb7d-7635-41ae-9dbd-5cb1267511d5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:39:02.610705 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1139/2284] 10.222.0.33 () {58 vars in 1232 bytes} [Mon Jul 25 05:39:02 2022] DELETE /identity/v3/projects/cf0ecccdccb146ca9ac25814bab43a75 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 05:39:12.134043 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:DELETE=8,keystone:SELECT=20,keystone:INSERT=1 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:39:12.604619 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=28,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:39:12.652854 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab9e0d44-0e86-4f97-a3fb-523e694cc519 admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:39:12.654304 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-ab9e0d44-0e86-4f97-a3fb-523e694cc519 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:39:12.671123 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab9e0d44-0e86-4f97-a3fb-523e694cc519 admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:39:12.738350 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01416bb2-6592-4bee-bce1-55a94604fdc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:39:12.739306 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01416bb2-6592-4bee-bce1-55a94604fdc8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:39:12.739612 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01416bb2-6592-4bee-bce1-55a94604fdc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:39:12.739885 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01416bb2-6592-4bee-bce1-55a94604fdc8 admin admin] PATH_INFO: `/v3/users/44f81ab87d404910aca84404a53579dc` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:39:12.741141 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01416bb2-6592-4bee-bce1-55a94604fdc8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=44f81ab87d404910aca84404a53579dc)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:39:12.743163 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01416bb2-6592-4bee-bce1-55a94604fdc8 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:39:12.793510 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-01416bb2-6592-4bee-bce1-55a94604fdc8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '44f81ab87d404910aca84404a53579dc'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:39:12.817534 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-01416bb2-6592-4bee-bce1-55a94604fdc8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '44f81ab87d404910aca84404a53579dc'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:39:12.818294 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-01416bb2-6592-4bee-bce1-55a94604fdc8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '44f81ab87d404910aca84404a53579dc'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:39:12.826408 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-01416bb2-6592-4bee-bce1-55a94604fdc8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '44f81ab87d404910aca84404a53579dc'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:39:12.827283 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-01416bb2-6592-4bee-bce1-55a94604fdc8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '44f81ab87d404910aca84404a53579dc'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:39:12.843736 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1146/2285] 10.222.0.33 () {58 vars in 1223 bytes} [Mon Jul 25 05:39:12 2022] DELETE /identity/v3/users/44f81ab87d404910aca84404a53579dc => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 05:39:12.845505 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c8cf07-aff5-4754-9f0c-d6f44fbe11a3 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:39:12.845926 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c8cf07-aff5-4754-9f0c-d6f44fbe11a3 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:39:12.846458 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c8cf07-aff5-4754-9f0c-d6f44fbe11a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:39:12.867966 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-65c8cf07-aff5-4754-9f0c-d6f44fbe11a3 None None] MFA Rules not processed for user `1eae540242d94c10b0ad33960a855f08`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:39:12.869926 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-65c8cf07-aff5-4754-9f0c-d6f44fbe11a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:39:12.954384 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1140/2286] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 25 05:39:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:39:12.965739 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01416bb2-6592-4bee-bce1-55a94604fdc8 admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:39:12.967028 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-01416bb2-6592-4bee-bce1-55a94604fdc8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:39:13.071904 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01416bb2-6592-4bee-bce1-55a94604fdc8 admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:39:13.076915 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b5e159-14a3-4efd-b0ef-8c0005ce4bc4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b2931200f99e43d79a50a9f6e02738e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92afba266cce4d1bad4134c351e74be3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:39:13.077592 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b5e159-14a3-4efd-b0ef-8c0005ce4bc4 service neutron] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:39:13.077874 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b5e159-14a3-4efd-b0ef-8c0005ce4bc4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:39:13.078147 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b5e159-14a3-4efd-b0ef-8c0005ce4bc4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:39:13.090465 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7b5e159-14a3-4efd-b0ef-8c0005ce4bc4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:39:13.092265 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7b5e159-14a3-4efd-b0ef-8c0005ce4bc4 service neutron] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:39:13.104863 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1147/2287] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jul 25 05:39:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 05:39:13.392402 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c8cf07-aff5-4754-9f0c-d6f44fbe11a3 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:39:13.393860 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-65c8cf07-aff5-4754-9f0c-d6f44fbe11a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:39:13.406697 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c8cf07-aff5-4754-9f0c-d6f44fbe11a3 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:39:13.413952 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e834b1d-f7da-4d11-b0fc-da4e07454f70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:39:13.415755 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e834b1d-f7da-4d11-b0fc-da4e07454f70 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:39:13.416038 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e834b1d-f7da-4d11-b0fc-da4e07454f70 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:39:13.416399 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e834b1d-f7da-4d11-b0fc-da4e07454f70 admin admin] PATH_INFO: `/v3/projects/c57e21c048594eed811473557c7f5694` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:39:13.417668 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e834b1d-f7da-4d11-b0fc-da4e07454f70 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c57e21c048594eed811473557c7f5694)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:39:13.420182 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e834b1d-f7da-4d11-b0fc-da4e07454f70 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:39:13.469014 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-7e834b1d-f7da-4d11-b0fc-da4e07454f70 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c57e21c048594eed811473557c7f5694', 'request_id': 'req-7e834b1d-f7da-4d11-b0fc-da4e07454f70'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:39:13.474663 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-7e834b1d-f7da-4d11-b0fc-da4e07454f70 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c57e21c048594eed811473557c7f5694', 'request_id': 'req-7e834b1d-f7da-4d11-b0fc-da4e07454f70'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:39:13.481034 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-7e834b1d-f7da-4d11-b0fc-da4e07454f70 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c57e21c048594eed811473557c7f5694', 'request_id': 'req-7e834b1d-f7da-4d11-b0fc-da4e07454f70'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:39:13.493165 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-7e834b1d-f7da-4d11-b0fc-da4e07454f70 admin admin] The token cache is being invalidate because project c57e21c048594eed811473557c7f5694 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80249) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 05:39:13.493479 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-7e834b1d-f7da-4d11-b0fc-da4e07454f70 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:39:13.494112 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-7e834b1d-f7da-4d11-b0fc-da4e07454f70 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:39:13.495693 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1141/2288] 10.222.0.33 () {58 vars in 1232 bytes} [Mon Jul 25 05:39:13 2022] DELETE /identity/v3/projects/c57e21c048594eed811473557c7f5694 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 05:39:13.501555 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed7284c-2f31-4f5e-bdc9-5d3fd7195694 None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:39:13.502042 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed7284c-2f31-4f5e-bdc9-5d3fd7195694 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:39:13.502147 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed7284c-2f31-4f5e-bdc9-5d3fd7195694 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:39:13.520924 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-3ed7284c-2f31-4f5e-bdc9-5d3fd7195694 None None] MFA Rules not processed for user `1eae540242d94c10b0ad33960a855f08`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:39:13.522135 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-3ed7284c-2f31-4f5e-bdc9-5d3fd7195694 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:39:13.538924 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:39:13.588298 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1148/2289] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 25 05:39:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:39:13.596151 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a45b9c-f099-474e-97ee-20c41f798fa1 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:39:13.596447 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a45b9c-f099-474e-97ee-20c41f798fa1 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:39:13.596734 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a45b9c-f099-474e-97ee-20c41f798fa1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:39:13.615209 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-c3a45b9c-f099-474e-97ee-20c41f798fa1 None None] MFA Rules not processed for user `1eae540242d94c10b0ad33960a855f08`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:39:13.616427 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-c3a45b9c-f099-474e-97ee-20c41f798fa1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:39:13.627104 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1142/2290] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 25 05:39:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:39:13.634149 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed7284c-2f31-4f5e-bdc9-5d3fd7195694 None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:39:13.649225 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed7284c-2f31-4f5e-bdc9-5d3fd7195694 None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:39:13.655310 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-009b129e-a54a-4535-a81c-ec1c7a59049b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:39:13.655986 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009b129e-a54a-4535-a81c-ec1c7a59049b admin admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:39:13.656278 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009b129e-a54a-4535-a81c-ec1c7a59049b admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:39:13.656559 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009b129e-a54a-4535-a81c-ec1c7a59049b admin admin] PATH_INFO: `/v3/domains` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:39:13.657371 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-009b129e-a54a-4535-a81c-ec1c7a59049b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:39:13.657651 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-009b129e-a54a-4535-a81c-ec1c7a59049b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:39:13.659308 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-009b129e-a54a-4535-a81c-ec1c7a59049b admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:39:13.671985 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1149/2291] 10.222.0.33 () {58 vars in 1152 bytes} [Mon Jul 25 05:39:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 05:39:13.676844 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a45b9c-f099-474e-97ee-20c41f798fa1 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:39:13.686519 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a45b9c-f099-474e-97ee-20c41f798fa1 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:39:13.691838 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d4dddd-ed78-471f-b738-2c01b4980972 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:39:13.692496 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d4dddd-ed78-471f-b738-2c01b4980972 admin admin] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:39:13.692803 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d4dddd-ed78-471f-b738-2c01b4980972 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:39:13.693088 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d4dddd-ed78-471f-b738-2c01b4980972 admin admin] PATH_INFO: `/v3/projects` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:39:13.693783 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14d4dddd-ed78-471f-b738-2c01b4980972 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:39:13.695679 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14d4dddd-ed78-471f-b738-2c01b4980972 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:39:13.736829 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1143/2292] 10.222.0.33 () {60 vars in 1152 bytes} [Mon Jul 25 05:39:13 2022] POST /identity/v3/projects => generated 383 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 05:39:13.741810 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-009b129e-a54a-4535-a81c-ec1c7a59049b admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:39:13.753906 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-009b129e-a54a-4535-a81c-ec1c7a59049b admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:39:13.819277 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e37f9a-c7f6-4250-ac67-38c295b8b013 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:39:13.819950 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e37f9a-c7f6-4250-ac67-38c295b8b013 admin admin] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:39:13.820239 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e37f9a-c7f6-4250-ac67-38c295b8b013 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:39:13.820521 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e37f9a-c7f6-4250-ac67-38c295b8b013 admin admin] PATH_INFO: `/v3/users` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:39:13.821259 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3e37f9a-c7f6-4250-ac67-38c295b8b013 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:39:13.823290 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3e37f9a-c7f6-4250-ac67-38c295b8b013 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:39:13.845585 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1150/2293] 10.222.0.33 () {60 vars in 1143 bytes} [Mon Jul 25 05:39:13 2022] POST /identity/v3/users => generated 292 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 05:39:13.849844 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d4dddd-ed78-471f-b738-2c01b4980972 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:39:13.860513 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d4dddd-ed78-471f-b738-2c01b4980972 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:39:13.866352 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27607591-b6a7-4039-88be-b0f48588a88b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:39:13.867045 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27607591-b6a7-4039-88be-b0f48588a88b admin admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:39:13.867359 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27607591-b6a7-4039-88be-b0f48588a88b admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:39:13.867639 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27607591-b6a7-4039-88be-b0f48588a88b admin admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:39:13.868371 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27607591-b6a7-4039-88be-b0f48588a88b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:39:13.870007 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27607591-b6a7-4039-88be-b0f48588a88b admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:39:13.881531 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1144/2294] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jul 25 05:39:13 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 05:39:13.885417 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e37f9a-c7f6-4250-ac67-38c295b8b013 admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:39:13.895736 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e37f9a-c7f6-4250-ac67-38c295b8b013 admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:39:13.900355 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51faee27-2d26-4153-973a-3b3882c65c47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:39:13.901048 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51faee27-2d26-4153-973a-3b3882c65c47 admin admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:39:13.901364 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51faee27-2d26-4153-973a-3b3882c65c47 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:39:13.901648 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51faee27-2d26-4153-973a-3b3882c65c47 admin admin] PATH_INFO: `/v3/roles` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:39:13.902379 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51faee27-2d26-4153-973a-3b3882c65c47 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:39:13.903974 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51faee27-2d26-4153-973a-3b3882c65c47 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:39:13.913989 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1151/2295] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jul 25 05:39:13 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 05:39:13.917910 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27607591-b6a7-4039-88be-b0f48588a88b admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:39:13.928793 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27607591-b6a7-4039-88be-b0f48588a88b admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:39:13.933414 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c195bf35-ac38-4e98-b3fe-d8ace9a870c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:39:13.934427 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c195bf35-ac38-4e98-b3fe-d8ace9a870c5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:39:13.934713 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c195bf35-ac38-4e98-b3fe-d8ace9a870c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:39:13.935053 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c195bf35-ac38-4e98-b3fe-d8ace9a870c5 admin admin] PATH_INFO: `/v3/projects/adeffdf0218141f58771a23acecee0f4/users/f13a76e545ac4d2f8c6bed5e79d88418/roles/5074e74e35f54d4b9bb415c34999a82a` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:39:13.952280 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c195bf35-ac38-4e98-b3fe-d8ace9a870c5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=adeffdf0218141f58771a23acecee0f4, user_id=f13a76e545ac4d2f8c6bed5e79d88418, role_id=5074e74e35f54d4b9bb415c34999a82a)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:39:13.955099 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c195bf35-ac38-4e98-b3fe-d8ace9a870c5 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:39:13.969554 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1145/2296] 10.222.0.33 () {60 vars in 1482 bytes} [Mon Jul 25 05:39:13 2022] PUT /identity/v3/projects/adeffdf0218141f58771a23acecee0f4/users/f13a76e545ac4d2f8c6bed5e79d88418/roles/5074e74e35f54d4b9bb415c34999a82a => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 05:39:13.980191 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c68098-2747-45ae-a7cd-0b12f66086ac None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:39:13.980420 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c68098-2747-45ae-a7cd-0b12f66086ac None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:39:13.980700 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c68098-2747-45ae-a7cd-0b12f66086ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:39:14.009266 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-63c68098-2747-45ae-a7cd-0b12f66086ac None None] MFA Rules not processed for user `f13a76e545ac4d2f8c6bed5e79d88418`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:39:14.010499 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-63c68098-2747-45ae-a7cd-0b12f66086ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:39:14.079856 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1152/2297] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 25 05:39:13 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:39:14.091560 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c195bf35-ac38-4e98-b3fe-d8ace9a870c5 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:39:14.093309 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-c195bf35-ac38-4e98-b3fe-d8ace9a870c5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:39:14.117320 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:39:14.174933 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c195bf35-ac38-4e98-b3fe-d8ace9a870c5 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:39:14.182855 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca2cbe0-8469-499c-bc6c-48e3d172ff96 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db57b478c25643dbaf936f90f6e17d0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92afba266cce4d1bad4134c351e74be3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:39:14.183618 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca2cbe0-8469-499c-bc6c-48e3d172ff96 service cinder] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:39:14.183918 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca2cbe0-8469-499c-bc6c-48e3d172ff96 service cinder] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:39:14.184200 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca2cbe0-8469-499c-bc6c-48e3d172ff96 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:39:14.194987 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aca2cbe0-8469-499c-bc6c-48e3d172ff96 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:39:14.196435 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aca2cbe0-8469-499c-bc6c-48e3d172ff96 service cinder] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:39:14.205975 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1146/2298] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jul 25 05:39:14 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 05:39:16.762007 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c68098-2747-45ae-a7cd-0b12f66086ac None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:39:16.793265 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:39:16.840804 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c68098-2747-45ae-a7cd-0b12f66086ac None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:39:16.848643 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f4d45a8-a4f0-4611-9b66-dca50b601a71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:39:16.849674 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4d45a8-a4f0-4611-9b66-dca50b601a71 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:39:16.849990 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4d45a8-a4f0-4611-9b66-dca50b601a71 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:39:16.850322 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4d45a8-a4f0-4611-9b66-dca50b601a71 admin admin] PATH_INFO: `/v3/users/f13a76e545ac4d2f8c6bed5e79d88418` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:39:16.851790 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f4d45a8-a4f0-4611-9b66-dca50b601a71 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f13a76e545ac4d2f8c6bed5e79d88418)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:39:16.854291 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f4d45a8-a4f0-4611-9b66-dca50b601a71 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:39:16.906584 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-1f4d45a8-a4f0-4611-9b66-dca50b601a71 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f13a76e545ac4d2f8c6bed5e79d88418'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:39:16.923607 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-1f4d45a8-a4f0-4611-9b66-dca50b601a71 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f13a76e545ac4d2f8c6bed5e79d88418'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:39:16.924444 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-1f4d45a8-a4f0-4611-9b66-dca50b601a71 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f13a76e545ac4d2f8c6bed5e79d88418'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:39:16.934092 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-1f4d45a8-a4f0-4611-9b66-dca50b601a71 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f13a76e545ac4d2f8c6bed5e79d88418'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:39:16.934823 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-1f4d45a8-a4f0-4611-9b66-dca50b601a71 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f13a76e545ac4d2f8c6bed5e79d88418'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:39:16.948276 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1153/2299] 10.222.0.33 () {58 vars in 1223 bytes} [Mon Jul 25 05:39:16 2022] DELETE /identity/v3/users/f13a76e545ac4d2f8c6bed5e79d88418 => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 05:39:16.953049 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c901cad-4c3a-4e73-a963-e95311385d05 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:39:16.953541 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c901cad-4c3a-4e73-a963-e95311385d05 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:39:16.954060 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c901cad-4c3a-4e73-a963-e95311385d05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:39:16.975982 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-3c901cad-4c3a-4e73-a963-e95311385d05 None None] MFA Rules not processed for user `1eae540242d94c10b0ad33960a855f08`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:39:16.977330 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-3c901cad-4c3a-4e73-a963-e95311385d05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:39:17.050369 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1147/2300] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 25 05:39:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:39:17.060691 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f4d45a8-a4f0-4611-9b66-dca50b601a71 admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:39:17.063127 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-1f4d45a8-a4f0-4611-9b66-dca50b601a71 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:39:17.126337 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f4d45a8-a4f0-4611-9b66-dca50b601a71 admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:39:17.130530 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a60754-81cd-4b41-8476-d94d2f3d0fa3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b2931200f99e43d79a50a9f6e02738e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92afba266cce4d1bad4134c351e74be3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:39:17.131181 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a60754-81cd-4b41-8476-d94d2f3d0fa3 service neutron] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:39:17.131466 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a60754-81cd-4b41-8476-d94d2f3d0fa3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:39:17.131742 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a60754-81cd-4b41-8476-d94d2f3d0fa3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:39:17.141153 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11a60754-81cd-4b41-8476-d94d2f3d0fa3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:39:17.142581 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11a60754-81cd-4b41-8476-d94d2f3d0fa3 service neutron] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:39:17.154172 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1154/2301] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jul 25 05:39:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 05:39:17.454404 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c901cad-4c3a-4e73-a963-e95311385d05 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:39:17.455873 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-3c901cad-4c3a-4e73-a963-e95311385d05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:39:17.466763 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cddd5dd-67a5-4ac1-bd7f-54c0557091d3 None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:39:17.467140 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cddd5dd-67a5-4ac1-bd7f-54c0557091d3 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:39:17.467515 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cddd5dd-67a5-4ac1-bd7f-54c0557091d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:39:17.469909 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c901cad-4c3a-4e73-a963-e95311385d05 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:39:17.475102 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c431b3-066a-4877-8710-a3b635133fa1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:39:17.476761 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c431b3-066a-4877-8710-a3b635133fa1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:39:17.477089 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c431b3-066a-4877-8710-a3b635133fa1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:39:17.477366 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c431b3-066a-4877-8710-a3b635133fa1 admin admin] PATH_INFO: `/v3/projects/adeffdf0218141f58771a23acecee0f4` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:39:17.478606 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40c431b3-066a-4877-8710-a3b635133fa1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=adeffdf0218141f58771a23acecee0f4)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:39:17.480534 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40c431b3-066a-4877-8710-a3b635133fa1 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:39:17.491126 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-3cddd5dd-67a5-4ac1-bd7f-54c0557091d3 None None] MFA Rules not processed for user `69f3276517de4f889dddd3180436ede9`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:39:17.495958 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-3cddd5dd-67a5-4ac1-bd7f-54c0557091d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:39:17.532282 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:39:17.539374 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-40c431b3-066a-4877-8710-a3b635133fa1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'adeffdf0218141f58771a23acecee0f4', 'request_id': 'req-40c431b3-066a-4877-8710-a3b635133fa1'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:39:17.545895 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-40c431b3-066a-4877-8710-a3b635133fa1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'adeffdf0218141f58771a23acecee0f4', 'request_id': 'req-40c431b3-066a-4877-8710-a3b635133fa1'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:39:17.555309 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-40c431b3-066a-4877-8710-a3b635133fa1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'adeffdf0218141f58771a23acecee0f4', 'request_id': 'req-40c431b3-066a-4877-8710-a3b635133fa1'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:39:17.558421 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1155/2302] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 25 05:39:17 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:39:17.579458 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cddd5dd-67a5-4ac1-bd7f-54c0557091d3 None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:39:17.581436 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-3cddd5dd-67a5-4ac1-bd7f-54c0557091d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:39:17.587129 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-40c431b3-066a-4877-8710-a3b635133fa1 admin admin] The token cache is being invalidate because project adeffdf0218141f58771a23acecee0f4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80249) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 05:39:17.587420 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-40c431b3-066a-4877-8710-a3b635133fa1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:39:17.588960 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-40c431b3-066a-4877-8710-a3b635133fa1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:39:17.590776 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1148/2303] 10.222.0.33 () {58 vars in 1232 bytes} [Mon Jul 25 05:39:17 2022] DELETE /identity/v3/projects/adeffdf0218141f58771a23acecee0f4 => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 05:39:17.597176 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c150c30-c8da-478c-8d9f-e5e3cae37a6b None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:39:17.598021 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c150c30-c8da-478c-8d9f-e5e3cae37a6b None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:39:17.598021 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c150c30-c8da-478c-8d9f-e5e3cae37a6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:39:17.600424 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cddd5dd-67a5-4ac1-bd7f-54c0557091d3 None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:39:17.620186 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-2c150c30-c8da-478c-8d9f-e5e3cae37a6b None None] MFA Rules not processed for user `1eae540242d94c10b0ad33960a855f08`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:39:17.621515 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-2c150c30-c8da-478c-8d9f-e5e3cae37a6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:39:17.683443 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:39:17.688412 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc95e21-4fdc-419f-b901-27606075fcf2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:39:17.689549 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc95e21-4fdc-419f-b901-27606075fcf2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:39:17.689925 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc95e21-4fdc-419f-b901-27606075fcf2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:39:17.690211 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc95e21-4fdc-419f-b901-27606075fcf2 admin admin] PATH_INFO: `/v3/users/c2ebc24a8b2f4db9ae7f7effa87cca5e` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:39:17.692020 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bc95e21-4fdc-419f-b901-27606075fcf2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c2ebc24a8b2f4db9ae7f7effa87cca5e)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:39:17.694229 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bc95e21-4fdc-419f-b901-27606075fcf2 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:39:17.707775 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1149/2304] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 25 05:39:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:39:17.717060 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5b2f02-776e-419b-a1a7-88d593e8d2a4 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:39:17.717643 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5b2f02-776e-419b-a1a7-88d593e8d2a4 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:39:17.718129 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5b2f02-776e-419b-a1a7-88d593e8d2a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:39:17.748241 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-ec5b2f02-776e-419b-a1a7-88d593e8d2a4 None None] MFA Rules not processed for user `1eae540242d94c10b0ad33960a855f08`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:39:17.749832 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-ec5b2f02-776e-419b-a1a7-88d593e8d2a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:39:17.759881 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1150/2305] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 25 05:39:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 05:39:17.764165 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec5b2f02-776e-419b-a1a7-88d593e8d2a4 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:39:17.767840 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-0bc95e21-4fdc-419f-b901-27606075fcf2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c2ebc24a8b2f4db9ae7f7effa87cca5e'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:39:17.775348 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec5b2f02-776e-419b-a1a7-88d593e8d2a4 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:39:17.780810 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03493eb4-e1ec-466e-bc65-ff20fb159cba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:39:17.781398 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03493eb4-e1ec-466e-bc65-ff20fb159cba admin admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:39:17.781677 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03493eb4-e1ec-466e-bc65-ff20fb159cba admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:39:17.781957 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03493eb4-e1ec-466e-bc65-ff20fb159cba admin admin] PATH_INFO: `/v3/domains` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:39:17.782703 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03493eb4-e1ec-466e-bc65-ff20fb159cba admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:39:17.783017 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03493eb4-e1ec-466e-bc65-ff20fb159cba admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:39:17.785651 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03493eb4-e1ec-466e-bc65-ff20fb159cba admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:39:17.789464 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-0bc95e21-4fdc-419f-b901-27606075fcf2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c2ebc24a8b2f4db9ae7f7effa87cca5e'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:39:17.790403 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-0bc95e21-4fdc-419f-b901-27606075fcf2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c2ebc24a8b2f4db9ae7f7effa87cca5e'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:39:17.799477 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1151/2306] 10.222.0.33 () {58 vars in 1152 bytes} [Mon Jul 25 05:39:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:39:17.811185 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03493eb4-e1ec-466e-bc65-ff20fb159cba admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:39:17.811185 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-03493eb4-e1ec-466e-bc65-ff20fb159cba admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:39:17.813570 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-0bc95e21-4fdc-419f-b901-27606075fcf2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c2ebc24a8b2f4db9ae7f7effa87cca5e'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:39:17.821080 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-0bc95e21-4fdc-419f-b901-27606075fcf2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c2ebc24a8b2f4db9ae7f7effa87cca5e'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:39:17.825426 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03493eb4-e1ec-466e-bc65-ff20fb159cba admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:39:17.830228 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dcfc07f-0fdb-4898-b449-0b7a6db32284 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:39:17.830950 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dcfc07f-0fdb-4898-b449-0b7a6db32284 admin admin] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:39:17.831343 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dcfc07f-0fdb-4898-b449-0b7a6db32284 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:39:17.831515 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dcfc07f-0fdb-4898-b449-0b7a6db32284 admin admin] PATH_INFO: `/v3/projects` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:39:17.833270 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dcfc07f-0fdb-4898-b449-0b7a6db32284 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:39:17.835220 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dcfc07f-0fdb-4898-b449-0b7a6db32284 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:39:17.837742 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1156/2307] 10.222.0.33 () {58 vars in 1223 bytes} [Mon Jul 25 05:39:17 2022] DELETE /identity/v3/users/c2ebc24a8b2f4db9ae7f7effa87cca5e => generated 0 bytes in 259 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 05:39:17.846106 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6af1f97-8214-471f-a9bb-34607941baad None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:39:17.846398 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6af1f97-8214-471f-a9bb-34607941baad None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:39:17.846679 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6af1f97-8214-471f-a9bb-34607941baad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:39:17.868014 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1152/2308] 10.222.0.33 () {60 vars in 1152 bytes} [Mon Jul 25 05:39:17 2022] POST /identity/v3/projects => generated 377 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 05:39:17.868912 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-f6af1f97-8214-471f-a9bb-34607941baad None None] MFA Rules not processed for user `1eae540242d94c10b0ad33960a855f08`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:39:17.869357 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dcfc07f-0fdb-4898-b449-0b7a6db32284 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:39:17.870652 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-f6af1f97-8214-471f-a9bb-34607941baad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:39:17.945899 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1157/2309] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 25 05:39:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:39:17.955348 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6af1f97-8214-471f-a9bb-34607941baad None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:39:17.956325 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dcfc07f-0fdb-4898-b449-0b7a6db32284 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:39:17.956899 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-f6af1f97-8214-471f-a9bb-34607941baad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:39:17.962200 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-472d4aec-3706-48aa-9909-353ad919a8a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:39:17.963101 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472d4aec-3706-48aa-9909-353ad919a8a7 admin admin] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:39:17.963464 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472d4aec-3706-48aa-9909-353ad919a8a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:39:17.963822 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472d4aec-3706-48aa-9909-353ad919a8a7 admin admin] PATH_INFO: `/v3/users` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:39:17.964366 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:39:17.964665 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-472d4aec-3706-48aa-9909-353ad919a8a7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:39:17.967000 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-472d4aec-3706-48aa-9909-353ad919a8a7 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:39:17.996525 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1153/2310] 10.222.0.33 () {60 vars in 1143 bytes} [Mon Jul 25 05:39:17 2022] POST /identity/v3/users => generated 289 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 25 05:39:18.000109 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-472d4aec-3706-48aa-9909-353ad919a8a7 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:39:18.014499 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-472d4aec-3706-48aa-9909-353ad919a8a7 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:39:18.020650 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd55e66-3076-468d-8a4b-5f789a661087 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:39:18.021381 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd55e66-3076-468d-8a4b-5f789a661087 admin admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:39:18.021770 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd55e66-3076-468d-8a4b-5f789a661087 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:39:18.022562 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd55e66-3076-468d-8a4b-5f789a661087 admin admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:39:18.023594 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bd55e66-3076-468d-8a4b-5f789a661087 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:39:18.027679 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bd55e66-3076-468d-8a4b-5f789a661087 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:39:18.043618 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1154/2311] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jul 25 05:39:17 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 25 05:39:18.046878 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6af1f97-8214-471f-a9bb-34607941baad None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:39:18.049466 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd55e66-3076-468d-8a4b-5f789a661087 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:39:18.051857 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb6c02e-fbae-43ec-a12c-80b62d811f76 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b2931200f99e43d79a50a9f6e02738e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92afba266cce4d1bad4134c351e74be3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:39:18.052524 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb6c02e-fbae-43ec-a12c-80b62d811f76 service neutron] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:39:18.052838 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb6c02e-fbae-43ec-a12c-80b62d811f76 service neutron] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:39:18.053128 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb6c02e-fbae-43ec-a12c-80b62d811f76 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:39:18.062555 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd55e66-3076-468d-8a4b-5f789a661087 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:39:18.063733 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efb6c02e-fbae-43ec-a12c-80b62d811f76 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:39:18.065220 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efb6c02e-fbae-43ec-a12c-80b62d811f76 service neutron] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:39:18.068172 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f5d99ae-eb8e-432d-88fe-09ef6879c8c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:39:18.069087 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5d99ae-eb8e-432d-88fe-09ef6879c8c0 admin admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:39:18.069465 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5d99ae-eb8e-432d-88fe-09ef6879c8c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:39:18.069818 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5d99ae-eb8e-432d-88fe-09ef6879c8c0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:39:18.070630 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f5d99ae-eb8e-432d-88fe-09ef6879c8c0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:39:18.072262 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f5d99ae-eb8e-432d-88fe-09ef6879c8c0 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:39:18.075668 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1158/2312] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jul 25 05:39:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 05:39:18.089006 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1155/2313] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jul 25 05:39:18 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 05:39:18.089930 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb6c02e-fbae-43ec-a12c-80b62d811f76 service neutron] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:39:18.101274 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb6c02e-fbae-43ec-a12c-80b62d811f76 service neutron] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:39:18.118285 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e5ed00-3ec3-4935-ae22-717c0fa3a350 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:39:18.119206 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e5ed00-3ec3-4935-ae22-717c0fa3a350 admin admin] REQUEST_METHOD: `PUT` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:39:18.119507 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e5ed00-3ec3-4935-ae22-717c0fa3a350 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:39:18.119769 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e5ed00-3ec3-4935-ae22-717c0fa3a350 admin admin] PATH_INFO: `/v3/projects/ee638472255a48d2bdffa17a4ff22339/users/1f815f111bf74739969736acea131927/roles/5074e74e35f54d4b9bb415c34999a82a` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:39:18.138143 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89e5ed00-3ec3-4935-ae22-717c0fa3a350 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ee638472255a48d2bdffa17a4ff22339, user_id=1f815f111bf74739969736acea131927, role_id=5074e74e35f54d4b9bb415c34999a82a)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:39:18.140838 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89e5ed00-3ec3-4935-ae22-717c0fa3a350 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:39:18.155873 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1159/2314] 10.222.0.33 () {60 vars in 1482 bytes} [Mon Jul 25 05:39:18 2022] PUT /identity/v3/projects/ee638472255a48d2bdffa17a4ff22339/users/1f815f111bf74739969736acea131927/roles/5074e74e35f54d4b9bb415c34999a82a => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 25 05:39:18.162004 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d32d57a-b34b-4ad7-b531-266bd04a4e24 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:39:18.162306 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d32d57a-b34b-4ad7-b531-266bd04a4e24 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:39:18.162590 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d32d57a-b34b-4ad7-b531-266bd04a4e24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:39:18.183036 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-5d32d57a-b34b-4ad7-b531-266bd04a4e24 None None] MFA Rules not processed for user `1eae540242d94c10b0ad33960a855f08`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:39:18.184324 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-5d32d57a-b34b-4ad7-b531-266bd04a4e24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:39:18.280839 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1156/2315] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 25 05:39:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:39:18.289137 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e5ed00-3ec3-4935-ae22-717c0fa3a350 admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:39:18.291095 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-89e5ed00-3ec3-4935-ae22-717c0fa3a350 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:39:18.358659 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e5ed00-3ec3-4935-ae22-717c0fa3a350 admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:39:18.367231 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3fefe74-8fcd-4427-b16d-e191154d2f77 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b2931200f99e43d79a50a9f6e02738e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92afba266cce4d1bad4134c351e74be3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:39:18.368119 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3fefe74-8fcd-4427-b16d-e191154d2f77 service neutron] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:39:18.368420 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3fefe74-8fcd-4427-b16d-e191154d2f77 service neutron] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:39:18.368958 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3fefe74-8fcd-4427-b16d-e191154d2f77 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:39:18.380177 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3fefe74-8fcd-4427-b16d-e191154d2f77 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:39:18.385654 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3fefe74-8fcd-4427-b16d-e191154d2f77 service neutron] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:39:18.400213 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1160/2316] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jul 25 05:39:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 05:39:18.416361 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d32d57a-b34b-4ad7-b531-266bd04a4e24 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:39:18.417940 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-5d32d57a-b34b-4ad7-b531-266bd04a4e24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:39:18.451220 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d32d57a-b34b-4ad7-b531-266bd04a4e24 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:39:18.467144 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df193b9-0218-4071-b898-a4c237b8274e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:39:18.468073 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df193b9-0218-4071-b898-a4c237b8274e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:39:18.468361 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df193b9-0218-4071-b898-a4c237b8274e admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:39:18.468697 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df193b9-0218-4071-b898-a4c237b8274e admin admin] PATH_INFO: `/v3/projects/f4e242dbc34a4321ac10a934cad9e5a0` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:39:18.470310 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2df193b9-0218-4071-b898-a4c237b8274e admin admin] RBAC: Authorizing `identity:delete_project(project_id=f4e242dbc34a4321ac10a934cad9e5a0)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:39:18.475060 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2df193b9-0218-4071-b898-a4c237b8274e admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:39:18.487274 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:39:18.559081 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-2df193b9-0218-4071-b898-a4c237b8274e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f4e242dbc34a4321ac10a934cad9e5a0', 'request_id': 'req-2df193b9-0218-4071-b898-a4c237b8274e'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:39:18.569254 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-2df193b9-0218-4071-b898-a4c237b8274e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f4e242dbc34a4321ac10a934cad9e5a0', 'request_id': 'req-2df193b9-0218-4071-b898-a4c237b8274e'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:39:18.576934 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-2df193b9-0218-4071-b898-a4c237b8274e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f4e242dbc34a4321ac10a934cad9e5a0', 'request_id': 'req-2df193b9-0218-4071-b898-a4c237b8274e'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:39:18.594726 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-2df193b9-0218-4071-b898-a4c237b8274e admin admin] The token cache is being invalidate because project f4e242dbc34a4321ac10a934cad9e5a0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80249) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 05:39:18.595094 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-2df193b9-0218-4071-b898-a4c237b8274e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:39:18.595915 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-2df193b9-0218-4071-b898-a4c237b8274e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:39:18.600861 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1157/2317] 10.222.0.33 () {58 vars in 1232 bytes} [Mon Jul 25 05:39:18 2022] DELETE /identity/v3/projects/f4e242dbc34a4321ac10a934cad9e5a0 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 05:39:18.603703 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c93df4-afc4-484d-8f4d-54093812a9af None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:39:18.604273 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c93df4-afc4-484d-8f4d-54093812a9af None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:39:18.604575 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c93df4-afc4-484d-8f4d-54093812a9af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:39:18.624849 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-00c93df4-afc4-484d-8f4d-54093812a9af None None] MFA Rules not processed for user `1eae540242d94c10b0ad33960a855f08`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:39:18.626172 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-00c93df4-afc4-484d-8f4d-54093812a9af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:39:18.711924 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1161/2318] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 25 05:39:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:39:18.720540 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e464331-3385-4990-b322-46511d1deaec None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:39:18.721269 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e464331-3385-4990-b322-46511d1deaec None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:39:18.721740 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e464331-3385-4990-b322-46511d1deaec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:39:18.755051 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-6e464331-3385-4990-b322-46511d1deaec None None] MFA Rules not processed for user `1eae540242d94c10b0ad33960a855f08`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:39:18.756616 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-6e464331-3385-4990-b322-46511d1deaec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:39:18.768071 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1158/2319] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 25 05:39:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:39:18.773857 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c93df4-afc4-484d-8f4d-54093812a9af None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:39:18.784673 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c93df4-afc4-484d-8f4d-54093812a9af None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:39:18.790065 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab74a951-3b2e-43ea-93fc-c28e2bb8f6ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:39:18.790738 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab74a951-3b2e-43ea-93fc-c28e2bb8f6ef admin admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:39:18.791032 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab74a951-3b2e-43ea-93fc-c28e2bb8f6ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:39:18.791309 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab74a951-3b2e-43ea-93fc-c28e2bb8f6ef admin admin] PATH_INFO: `/v3/domains` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:39:18.792547 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab74a951-3b2e-43ea-93fc-c28e2bb8f6ef admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:39:18.792547 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab74a951-3b2e-43ea-93fc-c28e2bb8f6ef admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:39:18.793981 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab74a951-3b2e-43ea-93fc-c28e2bb8f6ef admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:39:18.808249 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1162/2320] 10.222.0.33 () {58 vars in 1152 bytes} [Mon Jul 25 05:39:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:39:18.812910 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e464331-3385-4990-b322-46511d1deaec None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:39:18.825656 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e464331-3385-4990-b322-46511d1deaec None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:39:18.831061 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27eba652-5cc9-418a-a118-2b68904b6798 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:39:18.831556 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27eba652-5cc9-418a-a118-2b68904b6798 admin admin] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:39:18.832733 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27eba652-5cc9-418a-a118-2b68904b6798 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:39:18.833480 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27eba652-5cc9-418a-a118-2b68904b6798 admin admin] PATH_INFO: `/v3/projects` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:39:18.834304 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27eba652-5cc9-418a-a118-2b68904b6798 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:39:18.836253 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27eba652-5cc9-418a-a118-2b68904b6798 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:39:18.889899 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab74a951-3b2e-43ea-93fc-c28e2bb8f6ef admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:39:18.899151 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1159/2321] 10.222.0.33 () {60 vars in 1152 bytes} [Mon Jul 25 05:39:18 2022] POST /identity/v3/projects => generated 383 bytes in 78 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 05:39:18.907209 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab74a951-3b2e-43ea-93fc-c28e2bb8f6ef admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:39:18.932041 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:39:19.024374 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-893f2491-da88-45f4-bdfa-a1a4a759a466 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:39:19.025453 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893f2491-da88-45f4-bdfa-a1a4a759a466 admin admin] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:39:19.025935 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893f2491-da88-45f4-bdfa-a1a4a759a466 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:39:19.026415 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893f2491-da88-45f4-bdfa-a1a4a759a466 admin admin] PATH_INFO: `/v3/users` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:39:19.027325 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-893f2491-da88-45f4-bdfa-a1a4a759a466 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:39:19.029520 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-893f2491-da88-45f4-bdfa-a1a4a759a466 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:39:19.066867 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1163/2322] 10.222.0.33 () {60 vars in 1143 bytes} [Mon Jul 25 05:39:18 2022] POST /identity/v3/users => generated 292 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 05:39:19.069669 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27eba652-5cc9-418a-a118-2b68904b6798 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:39:19.087007 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27eba652-5cc9-418a-a118-2b68904b6798 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:39:19.122637 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ecf0d1-1c8d-4d38-bc8a-f535b4333435 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:39:19.123340 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ecf0d1-1c8d-4d38-bc8a-f535b4333435 admin admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:39:19.123732 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ecf0d1-1c8d-4d38-bc8a-f535b4333435 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:39:19.124094 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ecf0d1-1c8d-4d38-bc8a-f535b4333435 admin admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:39:19.124844 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01ecf0d1-1c8d-4d38-bc8a-f535b4333435 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:39:19.126469 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01ecf0d1-1c8d-4d38-bc8a-f535b4333435 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:39:19.139247 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1160/2323] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jul 25 05:39:19 2022] GET /identity/v3/roles => generated 1420 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 05:39:19.144890 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-893f2491-da88-45f4-bdfa-a1a4a759a466 admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:39:19.162040 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-893f2491-da88-45f4-bdfa-a1a4a759a466 admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:39:19.178292 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68bafc13-cc4c-4a65-9b1d-5fa07ed17424 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:39:19.179201 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68bafc13-cc4c-4a65-9b1d-5fa07ed17424 admin admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:39:19.179504 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68bafc13-cc4c-4a65-9b1d-5fa07ed17424 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:39:19.179782 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68bafc13-cc4c-4a65-9b1d-5fa07ed17424 admin admin] PATH_INFO: `/v3/roles` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:39:19.180707 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68bafc13-cc4c-4a65-9b1d-5fa07ed17424 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:39:19.182339 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68bafc13-cc4c-4a65-9b1d-5fa07ed17424 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:39:19.195559 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1164/2324] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jul 25 05:39:19 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 05:39:19.201722 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ecf0d1-1c8d-4d38-bc8a-f535b4333435 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:39:19.215473 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ecf0d1-1c8d-4d38-bc8a-f535b4333435 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:39:19.223094 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f846e0d-11ad-4495-a2f1-300964db5a6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:39:19.224173 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f846e0d-11ad-4495-a2f1-300964db5a6f admin admin] REQUEST_METHOD: `PUT` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:39:19.224900 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f846e0d-11ad-4495-a2f1-300964db5a6f admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:39:19.225394 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f846e0d-11ad-4495-a2f1-300964db5a6f admin admin] PATH_INFO: `/v3/projects/213b3114e35d4016b3169364760edb17/users/ce0944d760224239bd16aa0981db0eec/roles/5074e74e35f54d4b9bb415c34999a82a` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:39:19.247169 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f846e0d-11ad-4495-a2f1-300964db5a6f admin admin] RBAC: Authorizing `identity:create_grant(project_id=213b3114e35d4016b3169364760edb17, user_id=ce0944d760224239bd16aa0981db0eec, role_id=5074e74e35f54d4b9bb415c34999a82a)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:39:19.250198 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f846e0d-11ad-4495-a2f1-300964db5a6f admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:39:19.268350 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1161/2325] 10.222.0.33 () {60 vars in 1482 bytes} [Mon Jul 25 05:39:19 2022] PUT /identity/v3/projects/213b3114e35d4016b3169364760edb17/users/ce0944d760224239bd16aa0981db0eec/roles/5074e74e35f54d4b9bb415c34999a82a => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 05:39:19.276584 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97bcc4fe-e9d8-4f4c-ad94-51fd9f921afd None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:39:19.276933 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97bcc4fe-e9d8-4f4c-ad94-51fd9f921afd None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:39:19.277216 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97bcc4fe-e9d8-4f4c-ad94-51fd9f921afd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:39:19.301676 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-97bcc4fe-e9d8-4f4c-ad94-51fd9f921afd None None] MFA Rules not processed for user `1eae540242d94c10b0ad33960a855f08`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:39:19.303285 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-97bcc4fe-e9d8-4f4c-ad94-51fd9f921afd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:39:19.372376 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1165/2326] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 25 05:39:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:39:19.383884 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f846e0d-11ad-4495-a2f1-300964db5a6f admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:39:19.385990 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-7f846e0d-11ad-4495-a2f1-300964db5a6f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:39:19.513911 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f846e0d-11ad-4495-a2f1-300964db5a6f admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:39:19.526308 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8985233-8170-48c4-b4b3-5e403c04a064 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b2931200f99e43d79a50a9f6e02738e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92afba266cce4d1bad4134c351e74be3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:39:19.527211 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8985233-8170-48c4-b4b3-5e403c04a064 service neutron] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:39:19.527559 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8985233-8170-48c4-b4b3-5e403c04a064 service neutron] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:39:19.527845 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8985233-8170-48c4-b4b3-5e403c04a064 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:39:19.540162 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8985233-8170-48c4-b4b3-5e403c04a064 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:39:19.541953 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8985233-8170-48c4-b4b3-5e403c04a064 service neutron] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:39:19.559349 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1162/2327] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jul 25 05:39:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 176 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 05:39:21.406434 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367620d8-27d7-4304-ab57-8dffbba61b6b None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:39:21.407463 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367620d8-27d7-4304-ab57-8dffbba61b6b None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:39:21.407463 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367620d8-27d7-4304-ab57-8dffbba61b6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:39:21.469794 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-367620d8-27d7-4304-ab57-8dffbba61b6b None None] MFA Rules not processed for user `ce0944d760224239bd16aa0981db0eec`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:39:21.471859 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-367620d8-27d7-4304-ab57-8dffbba61b6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:39:21.609270 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1166/2328] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 25 05:39:21 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:39:21.620921 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8985233-8170-48c4-b4b3-5e403c04a064 service neutron] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:39:21.622357 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-d8985233-8170-48c4-b4b3-5e403c04a064 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:39:21.692421 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:39:21.728404 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8985233-8170-48c4-b4b3-5e403c04a064 service neutron] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:39:21.736001 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c8b3bf5-8a97-4b62-b753-7f8661377d9f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db57b478c25643dbaf936f90f6e17d0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92afba266cce4d1bad4134c351e74be3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:39:21.737030 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8b3bf5-8a97-4b62-b753-7f8661377d9f service cinder] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:39:21.737516 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8b3bf5-8a97-4b62-b753-7f8661377d9f service cinder] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:39:21.738133 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8b3bf5-8a97-4b62-b753-7f8661377d9f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:39:21.750461 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c8b3bf5-8a97-4b62-b753-7f8661377d9f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:39:21.752274 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c8b3bf5-8a97-4b62-b753-7f8661377d9f service cinder] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:39:21.765781 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1163/2329] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jul 25 05:39:21 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 25 05:39:23.564016 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4605a9d2-cb4a-4586-b434-5260e104fc36 None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:39:23.565174 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4605a9d2-cb4a-4586-b434-5260e104fc36 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:39:23.567447 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4605a9d2-cb4a-4586-b434-5260e104fc36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:39:23.605921 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-4605a9d2-cb4a-4586-b434-5260e104fc36 None None] MFA Rules not processed for user `ce0944d760224239bd16aa0981db0eec`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:39:23.610589 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-4605a9d2-cb4a-4586-b434-5260e104fc36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:39:23.627866 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1167/2330] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 25 05:39:23 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 65 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:39:24.303288 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364d5f8e-003c-478b-8e53-c384f312d2c1 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:39:24.303796 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364d5f8e-003c-478b-8e53-c384f312d2c1 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:39:24.303890 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364d5f8e-003c-478b-8e53-c384f312d2c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:39:24.323533 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-364d5f8e-003c-478b-8e53-c384f312d2c1 None None] MFA Rules not processed for user `37c1bb8500f74013b073febfc0d96217`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:39:24.324755 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-364d5f8e-003c-478b-8e53-c384f312d2c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:39:24.420864 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1164/2331] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 25 05:39:24 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:39:24.578677 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c428d30-3f5f-4ba1-b6c6-1cb478261ed6 None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:39:24.579198 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c428d30-3f5f-4ba1-b6c6-1cb478261ed6 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:39:24.579315 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c428d30-3f5f-4ba1-b6c6-1cb478261ed6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:39:24.636888 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-9c428d30-3f5f-4ba1-b6c6-1cb478261ed6 None None] MFA Rules not processed for user `1f815f111bf74739969736acea131927`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:39:24.642418 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-9c428d30-3f5f-4ba1-b6c6-1cb478261ed6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:39:24.649535 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:39:24.836580 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1168/2332] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 25 05:39:24 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 259 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:39:24.837501 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364d5f8e-003c-478b-8e53-c384f312d2c1 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:39:24.840391 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-364d5f8e-003c-478b-8e53-c384f312d2c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:39:24.859433 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364d5f8e-003c-478b-8e53-c384f312d2c1 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:39:24.864940 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e9fc4f4-f13a-4da7-98a6-8a1bf5f94057 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:39:24.865840 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9fc4f4-f13a-4da7-98a6-8a1bf5f94057 admin admin] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:39:24.866228 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9fc4f4-f13a-4da7-98a6-8a1bf5f94057 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:39:24.866622 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9fc4f4-f13a-4da7-98a6-8a1bf5f94057 admin admin] PATH_INFO: `/v3/projects` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:39:24.867426 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e9fc4f4-f13a-4da7-98a6-8a1bf5f94057 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:39:24.873823 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e9fc4f4-f13a-4da7-98a6-8a1bf5f94057 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:39:24.922854 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1165/2333] 10.222.0.33 () {60 vars in 1152 bytes} [Mon Jul 25 05:39:24 2022] POST /identity/v3/projects => generated 379 bytes in 86 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 25 05:39:24.928108 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c428d30-3f5f-4ba1-b6c6-1cb478261ed6 None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:39:24.950945 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c428d30-3f5f-4ba1-b6c6-1cb478261ed6 None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:39:25.040446 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ae716e-7fde-4ab1-aa56-a7a8e2a7e1a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:39:25.041181 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ae716e-7fde-4ab1-aa56-a7a8e2a7e1a8 admin admin] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:39:25.041481 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ae716e-7fde-4ab1-aa56-a7a8e2a7e1a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:39:25.041803 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ae716e-7fde-4ab1-aa56-a7a8e2a7e1a8 admin admin] PATH_INFO: `/v3/users` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:39:25.042521 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04ae716e-7fde-4ab1-aa56-a7a8e2a7e1a8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:39:25.044912 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04ae716e-7fde-4ab1-aa56-a7a8e2a7e1a8 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:39:25.076720 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1169/2334] 10.222.0.33 () {60 vars in 1143 bytes} [Mon Jul 25 05:39:24 2022] POST /identity/v3/users => generated 296 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 05:39:25.082543 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e9fc4f4-f13a-4da7-98a6-8a1bf5f94057 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:39:25.115504 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e9fc4f4-f13a-4da7-98a6-8a1bf5f94057 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:39:25.132654 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56483836-ea0c-4caa-b7d4-cab58aaf6005 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:39:25.133373 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56483836-ea0c-4caa-b7d4-cab58aaf6005 admin admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:39:25.133654 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56483836-ea0c-4caa-b7d4-cab58aaf6005 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:39:25.133926 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56483836-ea0c-4caa-b7d4-cab58aaf6005 admin admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:39:25.134672 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56483836-ea0c-4caa-b7d4-cab58aaf6005 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:39:25.136303 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56483836-ea0c-4caa-b7d4-cab58aaf6005 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:39:25.150019 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1166/2335] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jul 25 05:39:25 2022] GET /identity/v3/roles => generated 1420 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 05:39:25.154131 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ae716e-7fde-4ab1-aa56-a7a8e2a7e1a8 admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:39:25.168192 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ae716e-7fde-4ab1-aa56-a7a8e2a7e1a8 admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:39:25.179018 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3413805-c12d-4853-8ef9-ec093be22778 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:39:25.180218 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3413805-c12d-4853-8ef9-ec093be22778 admin admin] REQUEST_METHOD: `PUT` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:39:25.181108 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3413805-c12d-4853-8ef9-ec093be22778 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:39:25.181653 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3413805-c12d-4853-8ef9-ec093be22778 admin admin] PATH_INFO: `/v3/projects/e7503d2734414a67962cccd71e3b97e5/users/4b81270577ff4bb3a8b8b8479a59efd4/roles/39e5c666285a42fe8fc2c3afca6260d1` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:39:25.211420 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3413805-c12d-4853-8ef9-ec093be22778 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e7503d2734414a67962cccd71e3b97e5, user_id=4b81270577ff4bb3a8b8b8479a59efd4, role_id=39e5c666285a42fe8fc2c3afca6260d1)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:39:25.215025 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3413805-c12d-4853-8ef9-ec093be22778 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:39:25.246279 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1170/2336] 10.222.0.33 () {60 vars in 1482 bytes} [Mon Jul 25 05:39:25 2022] PUT /identity/v3/projects/e7503d2734414a67962cccd71e3b97e5/users/4b81270577ff4bb3a8b8b8479a59efd4/roles/39e5c666285a42fe8fc2c3afca6260d1 => generated 0 bytes in 92 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 25 05:39:25.250413 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56483836-ea0c-4caa-b7d4-cab58aaf6005 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:39:25.268230 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56483836-ea0c-4caa-b7d4-cab58aaf6005 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:39:25.351666 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e0a8672-050a-43b0-8248-cfb0c152b50e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:39:25.352627 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0a8672-050a-43b0-8248-cfb0c152b50e admin admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:39:25.353116 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0a8672-050a-43b0-8248-cfb0c152b50e admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:39:25.353456 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0a8672-050a-43b0-8248-cfb0c152b50e admin admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:39:25.354538 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e0a8672-050a-43b0-8248-cfb0c152b50e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:39:25.356328 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e0a8672-050a-43b0-8248-cfb0c152b50e admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:39:25.374480 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1167/2337] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jul 25 05:39:25 2022] GET /identity/v3/roles => generated 1420 bytes in 124 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 05:39:25.378193 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3413805-c12d-4853-8ef9-ec093be22778 admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:39:25.402073 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3413805-c12d-4853-8ef9-ec093be22778 admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:39:25.408467 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcac128e-adfe-4858-9d6f-45d1f937630f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:39:25.410379 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcac128e-adfe-4858-9d6f-45d1f937630f admin admin] REQUEST_METHOD: `PUT` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:39:25.410894 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcac128e-adfe-4858-9d6f-45d1f937630f admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:39:25.411355 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcac128e-adfe-4858-9d6f-45d1f937630f admin admin] PATH_INFO: `/v3/projects/e7503d2734414a67962cccd71e3b97e5/users/4b81270577ff4bb3a8b8b8479a59efd4/roles/39e5c666285a42fe8fc2c3afca6260d1` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:39:25.414499 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcac128e-adfe-4858-9d6f-45d1f937630f admin admin] RBAC: Authorizing `identity:create_grant(project_id=e7503d2734414a67962cccd71e3b97e5, user_id=4b81270577ff4bb3a8b8b8479a59efd4, role_id=39e5c666285a42fe8fc2c3afca6260d1)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:39:25.417767 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcac128e-adfe-4858-9d6f-45d1f937630f admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:39:25.439022 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1171/2338] 10.222.0.33 () {60 vars in 1482 bytes} [Mon Jul 25 05:39:25 2022] PUT /identity/v3/projects/e7503d2734414a67962cccd71e3b97e5/users/4b81270577ff4bb3a8b8b8479a59efd4/roles/39e5c666285a42fe8fc2c3afca6260d1 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 05:39:31.693489 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642978bc-0d28-40fe-b7ec-6bfdecc51ea1 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:39:31.694435 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642978bc-0d28-40fe-b7ec-6bfdecc51ea1 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:39:31.694787 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642978bc-0d28-40fe-b7ec-6bfdecc51ea1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:39:31.731029 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-642978bc-0d28-40fe-b7ec-6bfdecc51ea1 None None] MFA Rules not processed for user `4b81270577ff4bb3a8b8b8479a59efd4`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:39:31.734202 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-642978bc-0d28-40fe-b7ec-6bfdecc51ea1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:39:31.802464 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:39:31.854397 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1168/2339] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 25 05:39:31 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:39:31.868496 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcac128e-adfe-4858-9d6f-45d1f937630f admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:39:31.870312 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-dcac128e-adfe-4858-9d6f-45d1f937630f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:39:31.952929 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:39:31.961235 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcac128e-adfe-4858-9d6f-45d1f937630f admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:39:31.969693 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad3c913-33b7-4028-87e5-e8d5ba7bb0e7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cd482a61e8a420db45bb532e5dae6e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92afba266cce4d1bad4134c351e74be3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:39:31.970486 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad3c913-33b7-4028-87e5-e8d5ba7bb0e7 service nova] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:39:31.970486 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad3c913-33b7-4028-87e5-e8d5ba7bb0e7 service nova] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:39:31.970814 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad3c913-33b7-4028-87e5-e8d5ba7bb0e7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:39:32.040190 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cad3c913-33b7-4028-87e5-e8d5ba7bb0e7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:39:32.042001 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cad3c913-33b7-4028-87e5-e8d5ba7bb0e7 service nova] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:39:32.064991 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1172/2340] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jul 25 05:39:31 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 197 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 05:39:36.014040 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642978bc-0d28-40fe-b7ec-6bfdecc51ea1 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:39:36.015606 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-642978bc-0d28-40fe-b7ec-6bfdecc51ea1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:39:36.109534 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642978bc-0d28-40fe-b7ec-6bfdecc51ea1 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:39:36.124383 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e059b7-b5dd-487b-a19d-3175aff53b03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:39:36.125431 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e059b7-b5dd-487b-a19d-3175aff53b03 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:39:36.125719 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e059b7-b5dd-487b-a19d-3175aff53b03 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:39:36.125999 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e059b7-b5dd-487b-a19d-3175aff53b03 admin admin] PATH_INFO: `/v3/users/c0118c891dc74e30b9493b303e52db12` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:39:36.127237 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71e059b7-b5dd-487b-a19d-3175aff53b03 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c0118c891dc74e30b9493b303e52db12)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:39:36.129319 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71e059b7-b5dd-487b-a19d-3175aff53b03 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:39:36.194903 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-71e059b7-b5dd-487b-a19d-3175aff53b03 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c0118c891dc74e30b9493b303e52db12'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:39:36.195893 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-71e059b7-b5dd-487b-a19d-3175aff53b03 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c0118c891dc74e30b9493b303e52db12'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:39:36.203988 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-71e059b7-b5dd-487b-a19d-3175aff53b03 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c0118c891dc74e30b9493b303e52db12'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:39:36.223671 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-71e059b7-b5dd-487b-a19d-3175aff53b03 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c0118c891dc74e30b9493b303e52db12'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:39:36.224518 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-71e059b7-b5dd-487b-a19d-3175aff53b03 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c0118c891dc74e30b9493b303e52db12'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:39:36.244537 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1169/2341] 10.222.0.33 () {58 vars in 1223 bytes} [Mon Jul 25 05:39:36 2022] DELETE /identity/v3/users/c0118c891dc74e30b9493b303e52db12 => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 05:39:36.249959 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4983f8a5-5cfb-4d34-8b15-49f908f080ac None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:39:36.252718 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4983f8a5-5cfb-4d34-8b15-49f908f080ac None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:39:36.253030 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4983f8a5-5cfb-4d34-8b15-49f908f080ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:39:36.277122 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-4983f8a5-5cfb-4d34-8b15-49f908f080ac None None] MFA Rules not processed for user `1eae540242d94c10b0ad33960a855f08`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:39:36.278358 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-4983f8a5-5cfb-4d34-8b15-49f908f080ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:39:36.347710 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1173/2342] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 25 05:39:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:39:36.362549 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e059b7-b5dd-487b-a19d-3175aff53b03 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:39:36.365729 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-71e059b7-b5dd-487b-a19d-3175aff53b03 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:39:36.446733 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e059b7-b5dd-487b-a19d-3175aff53b03 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:39:36.450511 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ef697f-a3ad-4281-94f1-a47fe0455b88 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b2931200f99e43d79a50a9f6e02738e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92afba266cce4d1bad4134c351e74be3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:39:36.451172 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ef697f-a3ad-4281-94f1-a47fe0455b88 service neutron] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:39:36.451457 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ef697f-a3ad-4281-94f1-a47fe0455b88 service neutron] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:39:36.451784 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ef697f-a3ad-4281-94f1-a47fe0455b88 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:39:36.459226 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:39:36.470612 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0ef697f-a3ad-4281-94f1-a47fe0455b88 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:39:36.472551 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0ef697f-a3ad-4281-94f1-a47fe0455b88 service neutron] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:39:36.485133 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1170/2343] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jul 25 05:39:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 05:39:37.154962 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4983f8a5-5cfb-4d34-8b15-49f908f080ac None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:39:37.156668 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-4983f8a5-5cfb-4d34-8b15-49f908f080ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:39:37.184410 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4983f8a5-5cfb-4d34-8b15-49f908f080ac None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:39:37.212751 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdcb166d-8e1b-4e0d-9743-0ce11706294b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:39:37.213588 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdcb166d-8e1b-4e0d-9743-0ce11706294b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:39:37.213889 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdcb166d-8e1b-4e0d-9743-0ce11706294b admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:39:37.214159 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdcb166d-8e1b-4e0d-9743-0ce11706294b admin admin] PATH_INFO: `/v3/projects/9f4184b7029846d08de47f3b73346243` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:39:37.217858 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdcb166d-8e1b-4e0d-9743-0ce11706294b admin admin] RBAC: Authorizing `identity:delete_project(project_id=9f4184b7029846d08de47f3b73346243)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:39:37.219997 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdcb166d-8e1b-4e0d-9743-0ce11706294b admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:39:37.301026 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-cdcb166d-8e1b-4e0d-9743-0ce11706294b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9f4184b7029846d08de47f3b73346243', 'request_id': 'req-cdcb166d-8e1b-4e0d-9743-0ce11706294b'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:39:37.309345 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-cdcb166d-8e1b-4e0d-9743-0ce11706294b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9f4184b7029846d08de47f3b73346243', 'request_id': 'req-cdcb166d-8e1b-4e0d-9743-0ce11706294b'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:39:37.329069 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-cdcb166d-8e1b-4e0d-9743-0ce11706294b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9f4184b7029846d08de47f3b73346243', 'request_id': 'req-cdcb166d-8e1b-4e0d-9743-0ce11706294b'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:39:37.361390 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-cdcb166d-8e1b-4e0d-9743-0ce11706294b admin admin] The token cache is being invalidate because project 9f4184b7029846d08de47f3b73346243 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80248) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 05:39:37.361953 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-cdcb166d-8e1b-4e0d-9743-0ce11706294b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:39:37.364131 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-cdcb166d-8e1b-4e0d-9743-0ce11706294b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:39:37.368020 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1174/2344] 10.222.0.33 () {58 vars in 1232 bytes} [Mon Jul 25 05:39:37 2022] DELETE /identity/v3/projects/9f4184b7029846d08de47f3b73346243 => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 05:39:37.373866 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8888698-36d3-4cf1-a594-1fe6ba39299d None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:39:37.374731 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8888698-36d3-4cf1-a594-1fe6ba39299d None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:39:37.375259 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8888698-36d3-4cf1-a594-1fe6ba39299d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:39:37.409521 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-e8888698-36d3-4cf1-a594-1fe6ba39299d None None] MFA Rules not processed for user `1eae540242d94c10b0ad33960a855f08`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:39:37.411980 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-e8888698-36d3-4cf1-a594-1fe6ba39299d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:39:37.542042 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1171/2345] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 25 05:39:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:39:37.549372 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc18a34-cfce-49d7-9782-810504c7526c None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:39:37.550744 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc18a34-cfce-49d7-9782-810504c7526c None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:39:37.551400 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc18a34-cfce-49d7-9782-810504c7526c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:39:37.580645 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-8fc18a34-cfce-49d7-9782-810504c7526c None None] MFA Rules not processed for user `1eae540242d94c10b0ad33960a855f08`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:39:37.583653 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-8fc18a34-cfce-49d7-9782-810504c7526c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:39:37.602011 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8888698-36d3-4cf1-a594-1fe6ba39299d None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:39:37.609012 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1175/2346] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 25 05:39:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 05:39:37.628361 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8888698-36d3-4cf1-a594-1fe6ba39299d None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:39:37.637102 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b012e5ab-62d0-4aaa-8f5b-b74a2ae27bb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:39:37.637838 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b012e5ab-62d0-4aaa-8f5b-b74a2ae27bb5 admin admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:39:37.638131 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b012e5ab-62d0-4aaa-8f5b-b74a2ae27bb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:39:37.638411 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b012e5ab-62d0-4aaa-8f5b-b74a2ae27bb5 admin admin] PATH_INFO: `/v3/domains` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:39:37.639198 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b012e5ab-62d0-4aaa-8f5b-b74a2ae27bb5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:39:37.639477 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b012e5ab-62d0-4aaa-8f5b-b74a2ae27bb5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:39:37.642112 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b012e5ab-62d0-4aaa-8f5b-b74a2ae27bb5 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:39:37.676680 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1172/2347] 10.222.0.33 () {58 vars in 1152 bytes} [Mon Jul 25 05:39:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:39:37.678269 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc18a34-cfce-49d7-9782-810504c7526c None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:39:37.697829 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc18a34-cfce-49d7-9782-810504c7526c None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:39:37.708448 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a4e26a7-de06-4e03-9d7c-f60d3a9ed522 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:39:37.709932 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4e26a7-de06-4e03-9d7c-f60d3a9ed522 admin admin] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:39:37.710381 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4e26a7-de06-4e03-9d7c-f60d3a9ed522 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:39:37.711008 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4e26a7-de06-4e03-9d7c-f60d3a9ed522 admin admin] PATH_INFO: `/v3/projects` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:39:37.711987 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a4e26a7-de06-4e03-9d7c-f60d3a9ed522 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:39:37.715183 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a4e26a7-de06-4e03-9d7c-f60d3a9ed522 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:39:37.752523 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:39:37.803199 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1176/2348] 10.222.0.33 () {60 vars in 1152 bytes} [Mon Jul 25 05:39:37 2022] POST /identity/v3/projects => generated 373 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 05:39:37.803400 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b012e5ab-62d0-4aaa-8f5b-b74a2ae27bb5 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:39:37.826255 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b012e5ab-62d0-4aaa-8f5b-b74a2ae27bb5 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:39:37.931193 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc2457c3-483d-4786-ba66-13e554dc95d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:39:37.931915 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2457c3-483d-4786-ba66-13e554dc95d9 admin admin] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:39:37.932195 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2457c3-483d-4786-ba66-13e554dc95d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:39:37.932561 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2457c3-483d-4786-ba66-13e554dc95d9 admin admin] PATH_INFO: `/v3/users` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:39:37.933223 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc2457c3-483d-4786-ba66-13e554dc95d9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:39:37.935626 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc2457c3-483d-4786-ba66-13e554dc95d9 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:39:37.990104 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1173/2349] 10.222.0.33 () {60 vars in 1143 bytes} [Mon Jul 25 05:39:37 2022] POST /identity/v3/users => generated 287 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 05:39:37.992806 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a4e26a7-de06-4e03-9d7c-f60d3a9ed522 admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:39:38.024138 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a4e26a7-de06-4e03-9d7c-f60d3a9ed522 admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:39:38.043822 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5893d99-19ab-4140-aa00-6dcfac6890c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:39:38.044743 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5893d99-19ab-4140-aa00-6dcfac6890c1 admin admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:39:38.045167 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5893d99-19ab-4140-aa00-6dcfac6890c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:39:38.045548 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5893d99-19ab-4140-aa00-6dcfac6890c1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:39:38.046416 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5893d99-19ab-4140-aa00-6dcfac6890c1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:39:38.054898 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5893d99-19ab-4140-aa00-6dcfac6890c1 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:39:38.086573 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc2457c3-483d-4786-ba66-13e554dc95d9 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:39:38.090115 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1177/2350] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jul 25 05:39:37 2022] GET /identity/v3/roles => generated 1420 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 05:39:38.107415 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc2457c3-483d-4786-ba66-13e554dc95d9 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:39:38.124832 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17080ff1-2b99-40bf-8fd9-8d756de61dab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:39:38.125528 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17080ff1-2b99-40bf-8fd9-8d756de61dab admin admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:39:38.125816 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17080ff1-2b99-40bf-8fd9-8d756de61dab admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:39:38.128357 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17080ff1-2b99-40bf-8fd9-8d756de61dab admin admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:39:38.129775 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17080ff1-2b99-40bf-8fd9-8d756de61dab admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:39:38.134328 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17080ff1-2b99-40bf-8fd9-8d756de61dab admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:39:38.170065 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5893d99-19ab-4140-aa00-6dcfac6890c1 admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:39:38.173142 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1174/2351] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jul 25 05:39:38 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 05:39:38.190306 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5893d99-19ab-4140-aa00-6dcfac6890c1 admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:39:38.198750 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba88dd6d-a7cd-4f1f-8292-53f41cfc2266 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:39:38.199732 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba88dd6d-a7cd-4f1f-8292-53f41cfc2266 admin admin] REQUEST_METHOD: `PUT` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:39:38.200026 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba88dd6d-a7cd-4f1f-8292-53f41cfc2266 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:39:38.200354 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba88dd6d-a7cd-4f1f-8292-53f41cfc2266 admin admin] PATH_INFO: `/v3/projects/5bc77bdccdbf4a9f882b5badd02a437b/users/86b413ed0ab546adbd1d8d28a73496cc/roles/5074e74e35f54d4b9bb415c34999a82a` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:39:38.247495 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba88dd6d-a7cd-4f1f-8292-53f41cfc2266 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5bc77bdccdbf4a9f882b5badd02a437b, user_id=86b413ed0ab546adbd1d8d28a73496cc, role_id=5074e74e35f54d4b9bb415c34999a82a)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:39:38.250242 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba88dd6d-a7cd-4f1f-8292-53f41cfc2266 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:39:38.285469 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1178/2352] 10.222.0.33 () {60 vars in 1482 bytes} [Mon Jul 25 05:39:38 2022] PUT /identity/v3/projects/5bc77bdccdbf4a9f882b5badd02a437b/users/86b413ed0ab546adbd1d8d28a73496cc/roles/5074e74e35f54d4b9bb415c34999a82a => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 05:39:38.297307 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b41f6e4-864f-43b9-aeee-5d2fefcb5c2a None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:39:38.297604 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b41f6e4-864f-43b9-aeee-5d2fefcb5c2a None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:39:38.297892 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b41f6e4-864f-43b9-aeee-5d2fefcb5c2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:39:38.342758 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-7b41f6e4-864f-43b9-aeee-5d2fefcb5c2a None None] MFA Rules not processed for user `86b413ed0ab546adbd1d8d28a73496cc`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:39:38.344030 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-7b41f6e4-864f-43b9-aeee-5d2fefcb5c2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:39:38.423614 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:39:38.459107 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1175/2353] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 25 05:39:38 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:39:38.473333 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba88dd6d-a7cd-4f1f-8292-53f41cfc2266 admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:39:38.480225 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-ba88dd6d-a7cd-4f1f-8292-53f41cfc2266 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:39:38.601993 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba88dd6d-a7cd-4f1f-8292-53f41cfc2266 admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:39:38.607127 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f15eeee0-a05d-4bbf-88c9-e210b3881f4c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db57b478c25643dbaf936f90f6e17d0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92afba266cce4d1bad4134c351e74be3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:39:38.607797 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15eeee0-a05d-4bbf-88c9-e210b3881f4c service cinder] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:39:38.608086 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15eeee0-a05d-4bbf-88c9-e210b3881f4c service cinder] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:39:38.608368 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15eeee0-a05d-4bbf-88c9-e210b3881f4c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:39:38.620622 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f15eeee0-a05d-4bbf-88c9-e210b3881f4c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:39:38.622136 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f15eeee0-a05d-4bbf-88c9-e210b3881f4c service cinder] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:39:38.650399 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1179/2354] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jul 25 05:39:38 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 178 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 05:39:38.679833 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b41f6e4-864f-43b9-aeee-5d2fefcb5c2a None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:39:38.784417 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b41f6e4-864f-43b9-aeee-5d2fefcb5c2a None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:39:38.791316 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-821e8e5f-caf7-4405-b95f-6ee4685eee3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:39:38.791948 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821e8e5f-caf7-4405-b95f-6ee4685eee3f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:39:38.792307 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821e8e5f-caf7-4405-b95f-6ee4685eee3f admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:39:38.792658 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821e8e5f-caf7-4405-b95f-6ee4685eee3f admin admin] PATH_INFO: `/v3/users/86b413ed0ab546adbd1d8d28a73496cc` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:39:38.794220 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-821e8e5f-caf7-4405-b95f-6ee4685eee3f admin admin] RBAC: Authorizing `identity:delete_user(user_id=86b413ed0ab546adbd1d8d28a73496cc)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:39:38.799339 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-821e8e5f-caf7-4405-b95f-6ee4685eee3f admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:39:38.873251 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-821e8e5f-caf7-4405-b95f-6ee4685eee3f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '86b413ed0ab546adbd1d8d28a73496cc'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:39:38.873915 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-821e8e5f-caf7-4405-b95f-6ee4685eee3f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '86b413ed0ab546adbd1d8d28a73496cc'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:39:38.884127 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-821e8e5f-caf7-4405-b95f-6ee4685eee3f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '86b413ed0ab546adbd1d8d28a73496cc'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:39:38.904063 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-821e8e5f-caf7-4405-b95f-6ee4685eee3f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '86b413ed0ab546adbd1d8d28a73496cc'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:39:38.905174 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-821e8e5f-caf7-4405-b95f-6ee4685eee3f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '86b413ed0ab546adbd1d8d28a73496cc'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:39:38.926727 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1176/2355] 10.222.0.33 () {58 vars in 1223 bytes} [Mon Jul 25 05:39:38 2022] DELETE /identity/v3/users/86b413ed0ab546adbd1d8d28a73496cc => generated 0 bytes in 242 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 05:39:38.929428 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f04dc5-1753-429f-ae8e-50c59d0538a4 None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:39:38.929733 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f04dc5-1753-429f-ae8e-50c59d0538a4 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:39:38.930010 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f04dc5-1753-429f-ae8e-50c59d0538a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:39:38.951041 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-a1f04dc5-1753-429f-ae8e-50c59d0538a4 None None] MFA Rules not processed for user `1eae540242d94c10b0ad33960a855f08`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:39:38.952114 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-a1f04dc5-1753-429f-ae8e-50c59d0538a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:39:39.017174 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1180/2356] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 25 05:39:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:39:39.028073 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-821e8e5f-caf7-4405-b95f-6ee4685eee3f admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:39:39.029598 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-821e8e5f-caf7-4405-b95f-6ee4685eee3f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:39:39.115571 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-821e8e5f-caf7-4405-b95f-6ee4685eee3f admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:39:39.119483 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-220b3fe0-5ab7-40cb-b011-819609a019aa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b2931200f99e43d79a50a9f6e02738e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92afba266cce4d1bad4134c351e74be3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:39:39.120144 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220b3fe0-5ab7-40cb-b011-819609a019aa service neutron] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:39:39.120433 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220b3fe0-5ab7-40cb-b011-819609a019aa service neutron] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:39:39.120713 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220b3fe0-5ab7-40cb-b011-819609a019aa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:39:39.130261 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-220b3fe0-5ab7-40cb-b011-819609a019aa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:39:39.131699 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-220b3fe0-5ab7-40cb-b011-819609a019aa service neutron] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:39:39.141963 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1177/2357] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jul 25 05:39:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 05:39:39.444697 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f04dc5-1753-429f-ae8e-50c59d0538a4 None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:39:39.446272 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-a1f04dc5-1753-429f-ae8e-50c59d0538a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:39:39.461705 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f04dc5-1753-429f-ae8e-50c59d0538a4 None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:39:39.465979 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c4283d-e76c-4a6f-ac15-bfa4de610955 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:39:39.467637 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c4283d-e76c-4a6f-ac15-bfa4de610955 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:39:39.468034 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c4283d-e76c-4a6f-ac15-bfa4de610955 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:39:39.468414 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c4283d-e76c-4a6f-ac15-bfa4de610955 admin admin] PATH_INFO: `/v3/projects/5bc77bdccdbf4a9f882b5badd02a437b` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:39:39.469823 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0c4283d-e76c-4a6f-ac15-bfa4de610955 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5bc77bdccdbf4a9f882b5badd02a437b)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:39:39.471855 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0c4283d-e76c-4a6f-ac15-bfa4de610955 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:39:39.530823 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-d0c4283d-e76c-4a6f-ac15-bfa4de610955 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5bc77bdccdbf4a9f882b5badd02a437b', 'request_id': 'req-d0c4283d-e76c-4a6f-ac15-bfa4de610955'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:39:39.532836 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=93,keystone:INSERT=2,keystone:DELETE=5 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:39:39.537607 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-d0c4283d-e76c-4a6f-ac15-bfa4de610955 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5bc77bdccdbf4a9f882b5badd02a437b', 'request_id': 'req-d0c4283d-e76c-4a6f-ac15-bfa4de610955'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:39:39.545025 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-d0c4283d-e76c-4a6f-ac15-bfa4de610955 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5bc77bdccdbf4a9f882b5badd02a437b', 'request_id': 'req-d0c4283d-e76c-4a6f-ac15-bfa4de610955'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:39:39.557851 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-d0c4283d-e76c-4a6f-ac15-bfa4de610955 admin admin] The token cache is being invalidate because project 5bc77bdccdbf4a9f882b5badd02a437b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80248) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 05:39:39.558070 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-d0c4283d-e76c-4a6f-ac15-bfa4de610955 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:39:39.558679 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-d0c4283d-e76c-4a6f-ac15-bfa4de610955 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:39:39.560257 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1181/2358] 10.222.0.33 () {58 vars in 1232 bytes} [Mon Jul 25 05:39:39 2022] DELETE /identity/v3/projects/5bc77bdccdbf4a9f882b5badd02a437b => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 05:39:39.565759 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a673bc-2d6f-4b01-81eb-a8bf6d5d8c20 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:39:39.566908 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a673bc-2d6f-4b01-81eb-a8bf6d5d8c20 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:39:39.567014 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a673bc-2d6f-4b01-81eb-a8bf6d5d8c20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:39:39.585541 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-c5a673bc-2d6f-4b01-81eb-a8bf6d5d8c20 None None] MFA Rules not processed for user `1eae540242d94c10b0ad33960a855f08`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:39:39.587053 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-c5a673bc-2d6f-4b01-81eb-a8bf6d5d8c20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:39:39.592682 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:39:39.652304 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1178/2359] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 25 05:39:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:39:39.662018 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b7f9ee-47e2-41c2-9aa8-de6f10079bee None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:39:39.662542 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b7f9ee-47e2-41c2-9aa8-de6f10079bee None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:39:39.662979 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b7f9ee-47e2-41c2-9aa8-de6f10079bee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:39:39.687983 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-49b7f9ee-47e2-41c2-9aa8-de6f10079bee None None] MFA Rules not processed for user `1eae540242d94c10b0ad33960a855f08`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:39:39.689243 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-49b7f9ee-47e2-41c2-9aa8-de6f10079bee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:39:39.697872 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1182/2360] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 25 05:39:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:39:39.702681 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a673bc-2d6f-4b01-81eb-a8bf6d5d8c20 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:39:39.713402 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a673bc-2d6f-4b01-81eb-a8bf6d5d8c20 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:39:39.718068 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a05cc1-f876-4f23-bc97-56f454ff8e52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:39:39.718723 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a05cc1-f876-4f23-bc97-56f454ff8e52 admin admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:39:39.719004 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a05cc1-f876-4f23-bc97-56f454ff8e52 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:39:39.719288 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a05cc1-f876-4f23-bc97-56f454ff8e52 admin admin] PATH_INFO: `/v3/domains` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:39:39.720012 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0a05cc1-f876-4f23-bc97-56f454ff8e52 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:39:39.720319 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0a05cc1-f876-4f23-bc97-56f454ff8e52 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:39:39.721940 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0a05cc1-f876-4f23-bc97-56f454ff8e52 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:39:39.735150 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1179/2361] 10.222.0.33 () {58 vars in 1152 bytes} [Mon Jul 25 05:39:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:39:39.739684 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b7f9ee-47e2-41c2-9aa8-de6f10079bee None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:39:39.749893 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b7f9ee-47e2-41c2-9aa8-de6f10079bee None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:39:39.755094 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56860f6f-7318-4474-a81b-260c0772115e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:39:39.755755 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56860f6f-7318-4474-a81b-260c0772115e admin admin] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:39:39.756037 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56860f6f-7318-4474-a81b-260c0772115e admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:39:39.756310 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56860f6f-7318-4474-a81b-260c0772115e admin admin] PATH_INFO: `/v3/projects` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:39:39.757058 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56860f6f-7318-4474-a81b-260c0772115e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:39:39.758923 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56860f6f-7318-4474-a81b-260c0772115e admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:39:39.788345 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1183/2362] 10.222.0.33 () {60 vars in 1152 bytes} [Mon Jul 25 05:39:39 2022] POST /identity/v3/projects => generated 373 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 05:39:39.793121 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a05cc1-f876-4f23-bc97-56f454ff8e52 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:39:39.804610 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a05cc1-f876-4f23-bc97-56f454ff8e52 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:39:39.878311 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-375a8039-b60c-49fe-8fd7-2c52a5cc1362 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:39:39.879589 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375a8039-b60c-49fe-8fd7-2c52a5cc1362 admin admin] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:39:39.879878 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375a8039-b60c-49fe-8fd7-2c52a5cc1362 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:39:39.880161 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375a8039-b60c-49fe-8fd7-2c52a5cc1362 admin admin] PATH_INFO: `/v3/users` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:39:39.880903 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-375a8039-b60c-49fe-8fd7-2c52a5cc1362 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:39:39.882841 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-375a8039-b60c-49fe-8fd7-2c52a5cc1362 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:39:39.906307 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1180/2363] 10.222.0.33 () {60 vars in 1143 bytes} [Mon Jul 25 05:39:39 2022] POST /identity/v3/users => generated 287 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 05:39:39.907522 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56860f6f-7318-4474-a81b-260c0772115e admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:39:39.918419 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56860f6f-7318-4474-a81b-260c0772115e admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:39:39.923145 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-192f1282-2ab2-4bc9-8c1f-221074f3e9ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:39:39.923826 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192f1282-2ab2-4bc9-8c1f-221074f3e9ba admin admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:39:39.924129 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192f1282-2ab2-4bc9-8c1f-221074f3e9ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:39:39.924395 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192f1282-2ab2-4bc9-8c1f-221074f3e9ba admin admin] PATH_INFO: `/v3/roles` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:39:39.925196 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-192f1282-2ab2-4bc9-8c1f-221074f3e9ba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:39:39.926898 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-192f1282-2ab2-4bc9-8c1f-221074f3e9ba admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:39:39.943630 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1184/2364] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jul 25 05:39:39 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 05:39:39.945950 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-375a8039-b60c-49fe-8fd7-2c52a5cc1362 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:39:39.957127 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-375a8039-b60c-49fe-8fd7-2c52a5cc1362 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:39:39.962356 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d01390-036a-45ce-b3c5-d836b0959ba0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:39:39.963105 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d01390-036a-45ce-b3c5-d836b0959ba0 admin admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:39:39.963466 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d01390-036a-45ce-b3c5-d836b0959ba0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:39:39.963820 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d01390-036a-45ce-b3c5-d836b0959ba0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:39:39.964632 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93d01390-036a-45ce-b3c5-d836b0959ba0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:39:39.966504 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93d01390-036a-45ce-b3c5-d836b0959ba0 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:39:39.980547 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1181/2365] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jul 25 05:39:39 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 05:39:39.982065 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-192f1282-2ab2-4bc9-8c1f-221074f3e9ba admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:39:39.993087 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-192f1282-2ab2-4bc9-8c1f-221074f3e9ba admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:39:39.997895 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1fbe08b-a3c3-4f32-aa16-a3606c4bda59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:39:39.998881 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1fbe08b-a3c3-4f32-aa16-a3606c4bda59 admin admin] REQUEST_METHOD: `PUT` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:39:39.999175 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1fbe08b-a3c3-4f32-aa16-a3606c4bda59 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:39:39.999460 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1fbe08b-a3c3-4f32-aa16-a3606c4bda59 admin admin] PATH_INFO: `/v3/projects/5d8e26c74f99499fb3890a0613364236/users/08384d4a71da47f380bd748acd913e27/roles/5074e74e35f54d4b9bb415c34999a82a` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:39:40.016472 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1fbe08b-a3c3-4f32-aa16-a3606c4bda59 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5d8e26c74f99499fb3890a0613364236, user_id=08384d4a71da47f380bd748acd913e27, role_id=5074e74e35f54d4b9bb415c34999a82a)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:39:40.019215 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1fbe08b-a3c3-4f32-aa16-a3606c4bda59 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:39:40.037890 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1185/2366] 10.222.0.33 () {60 vars in 1482 bytes} [Mon Jul 25 05:39:39 2022] PUT /identity/v3/projects/5d8e26c74f99499fb3890a0613364236/users/08384d4a71da47f380bd748acd913e27/roles/5074e74e35f54d4b9bb415c34999a82a => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 05:39:40.048006 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59409ac-2cc4-4f71-8585-7e2f225f6341 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:39:40.048302 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59409ac-2cc4-4f71-8585-7e2f225f6341 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:39:40.048584 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59409ac-2cc4-4f71-8585-7e2f225f6341 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:39:40.076593 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-a59409ac-2cc4-4f71-8585-7e2f225f6341 None None] MFA Rules not processed for user `08384d4a71da47f380bd748acd913e27`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:39:40.077775 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-a59409ac-2cc4-4f71-8585-7e2f225f6341 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:39:40.153315 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1182/2367] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 25 05:39:40 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:39:40.165329 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1fbe08b-a3c3-4f32-aa16-a3606c4bda59 admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:39:40.167706 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-c1fbe08b-a3c3-4f32-aa16-a3606c4bda59 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:39:40.228203 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1fbe08b-a3c3-4f32-aa16-a3606c4bda59 admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:39:40.234940 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e91e6075-1fa0-44b9-a00e-98cdea38b371 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db57b478c25643dbaf936f90f6e17d0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92afba266cce4d1bad4134c351e74be3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:39:40.235632 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91e6075-1fa0-44b9-a00e-98cdea38b371 service cinder] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:39:40.235926 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91e6075-1fa0-44b9-a00e-98cdea38b371 service cinder] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:39:40.236211 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91e6075-1fa0-44b9-a00e-98cdea38b371 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:39:40.246542 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e91e6075-1fa0-44b9-a00e-98cdea38b371 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:39:40.247974 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e91e6075-1fa0-44b9-a00e-98cdea38b371 service cinder] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:39:40.258151 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1186/2368] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jul 25 05:39:40 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 05:39:41.617578 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6268ef25-8f8f-41d4-806a-7d08b18dea14 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:39:41.618116 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6268ef25-8f8f-41d4-806a-7d08b18dea14 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:39:41.618116 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6268ef25-8f8f-41d4-806a-7d08b18dea14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:39:41.630904 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:39:41.640385 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-6268ef25-8f8f-41d4-806a-7d08b18dea14 None None] MFA Rules not processed for user `08384d4a71da47f380bd748acd913e27`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:39:41.640385 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-6268ef25-8f8f-41d4-806a-7d08b18dea14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:39:41.647917 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1183/2369] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 25 05:39:41 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:39:50.252799 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=59,keystone:DELETE=2,keystone:INSERT=3 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:39:51.487896 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa0b429-2dbe-41c5-9595-c4899c1f670b None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:39:51.488574 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa0b429-2dbe-41c5-9595-c4899c1f670b None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:39:51.488574 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa0b429-2dbe-41c5-9595-c4899c1f670b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:39:51.507191 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-caa0b429-2dbe-41c5-9595-c4899c1f670b None None] MFA Rules not processed for user `1f815f111bf74739969736acea131927`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:39:51.508619 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-caa0b429-2dbe-41c5-9595-c4899c1f670b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:39:51.568358 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1187/2370] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 25 05:39:51 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:39:58.042454 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72489e90-eb4d-48ca-8977-9fa6be5ee13c None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:39:58.042961 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72489e90-eb4d-48ca-8977-9fa6be5ee13c None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:39:58.043066 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72489e90-eb4d-48ca-8977-9fa6be5ee13c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:39:58.069291 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-72489e90-eb4d-48ca-8977-9fa6be5ee13c None None] MFA Rules not processed for user `3352531be1b54f76b864383ccc2e5ec2`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:39:58.070763 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-72489e90-eb4d-48ca-8977-9fa6be5ee13c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:39:58.189048 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1184/2371] 10.222.0.33 () {58 vars in 1009 bytes} [Mon Jul 25 05:39:58 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:39:58.262663 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa0b429-2dbe-41c5-9595-c4899c1f670b None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:39:58.264728 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-caa0b429-2dbe-41c5-9595-c4899c1f670b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:39:58.352395 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa0b429-2dbe-41c5-9595-c4899c1f670b None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:39:58.359150 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7efa0235-e84b-4e1b-aa72-7425ff834814 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb135a8d5d7444cfbfa336115ba2ba44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92afba266cce4d1bad4134c351e74be3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:39:58.359819 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7efa0235-e84b-4e1b-aa72-7425ff834814 service swift] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:39:58.360109 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7efa0235-e84b-4e1b-aa72-7425ff834814 service swift] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:39:58.360389 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7efa0235-e84b-4e1b-aa72-7425ff834814 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:39:58.371509 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7efa0235-e84b-4e1b-aa72-7425ff834814 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:39:58.373321 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7efa0235-e84b-4e1b-aa72-7425ff834814 service swift] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:39:58.387007 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1188/2372] 10.222.0.33 () {60 vars in 1384 bytes} [Mon Jul 25 05:39:58 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 25 05:39:58.938870 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd427961-aaea-4d2a-a241-1cc7419757ce None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:39:58.939169 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd427961-aaea-4d2a-a241-1cc7419757ce None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:39:58.939458 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd427961-aaea-4d2a-a241-1cc7419757ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:39:58.958740 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-dd427961-aaea-4d2a-a241-1cc7419757ce None None] MFA Rules not processed for user `3352531be1b54f76b864383ccc2e5ec2`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:39:58.962692 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-dd427961-aaea-4d2a-a241-1cc7419757ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:39:58.971513 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1185/2373] 10.222.0.33 () {58 vars in 967 bytes} [Mon Jul 25 05:39:58 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:39:58.985772 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7efa0235-e84b-4e1b-aa72-7425ff834814 service swift] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:39:58.996591 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7efa0235-e84b-4e1b-aa72-7425ff834814 service swift] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:39:59.001935 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa138be-e0c6-43f9-ac30-bf7947e5446c service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb135a8d5d7444cfbfa336115ba2ba44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92afba266cce4d1bad4134c351e74be3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:39:59.002760 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa138be-e0c6-43f9-ac30-bf7947e5446c service swift] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:39:59.003116 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa138be-e0c6-43f9-ac30-bf7947e5446c service swift] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:39:59.003577 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa138be-e0c6-43f9-ac30-bf7947e5446c service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:39:59.012735 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fa138be-e0c6-43f9-ac30-bf7947e5446c service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:39:59.014335 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fa138be-e0c6-43f9-ac30-bf7947e5446c service swift] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:39:59.023666 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1189/2374] 10.222.0.33 () {60 vars in 1384 bytes} [Mon Jul 25 05:39:58 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 25 05:39:59.279911 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6f762b-2664-4c92-b1cb-a3a0c62a4241 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:39:59.280418 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6f762b-2664-4c92-b1cb-a3a0c62a4241 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:39:59.280530 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6f762b-2664-4c92-b1cb-a3a0c62a4241 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:39:59.299258 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-df6f762b-2664-4c92-b1cb-a3a0c62a4241 None None] MFA Rules not processed for user `3352531be1b54f76b864383ccc2e5ec2`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:39:59.300471 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-df6f762b-2664-4c92-b1cb-a3a0c62a4241 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:39:59.310301 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1186/2375] 10.222.0.33 () {58 vars in 1009 bytes} [Mon Jul 25 05:39:59 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:39:59.330974 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa138be-e0c6-43f9-ac30-bf7947e5446c service swift] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:39:59.342380 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa138be-e0c6-43f9-ac30-bf7947e5446c service swift] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:39:59.347405 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d749c9d2-f8e0-429f-9018-8f14dc2b81f3 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb135a8d5d7444cfbfa336115ba2ba44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92afba266cce4d1bad4134c351e74be3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:39:59.348215 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d749c9d2-f8e0-429f-9018-8f14dc2b81f3 service swift] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:39:59.348503 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d749c9d2-f8e0-429f-9018-8f14dc2b81f3 service swift] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:39:59.348801 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d749c9d2-f8e0-429f-9018-8f14dc2b81f3 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:39:59.359792 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d749c9d2-f8e0-429f-9018-8f14dc2b81f3 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:39:59.361464 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d749c9d2-f8e0-429f-9018-8f14dc2b81f3 service swift] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:39:59.373634 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1190/2376] 10.222.0.33 () {60 vars in 1384 bytes} [Mon Jul 25 05:39:59 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 25 05:39:59.699670 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ac8d9a-83a0-4333-b336-5d5c41a082e7 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:39:59.700016 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ac8d9a-83a0-4333-b336-5d5c41a082e7 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:39:59.700299 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ac8d9a-83a0-4333-b336-5d5c41a082e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:39:59.718251 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-a3ac8d9a-83a0-4333-b336-5d5c41a082e7 None None] MFA Rules not processed for user `3352531be1b54f76b864383ccc2e5ec2`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:39:59.719447 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-a3ac8d9a-83a0-4333-b336-5d5c41a082e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:39:59.727126 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1187/2377] 10.222.0.33 () {58 vars in 967 bytes} [Mon Jul 25 05:39:59 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:39:59.738995 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d749c9d2-f8e0-429f-9018-8f14dc2b81f3 service swift] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:39:59.749514 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d749c9d2-f8e0-429f-9018-8f14dc2b81f3 service swift] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:39:59.754491 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da0cc46a-27e2-483c-b786-4a5faa036b57 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb135a8d5d7444cfbfa336115ba2ba44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92afba266cce4d1bad4134c351e74be3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:39:59.755127 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0cc46a-27e2-483c-b786-4a5faa036b57 service swift] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:39:59.755413 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0cc46a-27e2-483c-b786-4a5faa036b57 service swift] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:39:59.755691 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0cc46a-27e2-483c-b786-4a5faa036b57 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:39:59.763845 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da0cc46a-27e2-483c-b786-4a5faa036b57 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:39:59.765459 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da0cc46a-27e2-483c-b786-4a5faa036b57 service swift] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:39:59.773997 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1191/2378] 10.222.0.33 () {60 vars in 1384 bytes} [Mon Jul 25 05:39:59 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 25 05:40:01.875000 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ac8d9a-83a0-4333-b336-5d5c41a082e7 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:01.937528 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ac8d9a-83a0-4333-b336-5d5c41a082e7 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:01.942357 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b5684b-fc06-441c-8684-22ca5489f310 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:01.943227 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b5684b-fc06-441c-8684-22ca5489f310 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:01.943517 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b5684b-fc06-441c-8684-22ca5489f310 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:01.943839 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b5684b-fc06-441c-8684-22ca5489f310 admin admin] PATH_INFO: `/v3/users/08384d4a71da47f380bd748acd913e27` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:01.945066 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98b5684b-fc06-441c-8684-22ca5489f310 admin admin] RBAC: Authorizing `identity:delete_user(user_id=08384d4a71da47f380bd748acd913e27)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:40:01.947008 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98b5684b-fc06-441c-8684-22ca5489f310 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:40:01.995645 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-98b5684b-fc06-441c-8684-22ca5489f310 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '08384d4a71da47f380bd748acd913e27'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:40:01.996302 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-98b5684b-fc06-441c-8684-22ca5489f310 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '08384d4a71da47f380bd748acd913e27'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:40:02.003076 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-98b5684b-fc06-441c-8684-22ca5489f310 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '08384d4a71da47f380bd748acd913e27'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:40:02.014633 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-98b5684b-fc06-441c-8684-22ca5489f310 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '08384d4a71da47f380bd748acd913e27'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:40:02.015303 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-98b5684b-fc06-441c-8684-22ca5489f310 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '08384d4a71da47f380bd748acd913e27'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:40:02.028538 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1188/2379] 10.222.0.33 () {58 vars in 1223 bytes} [Mon Jul 25 05:40:01 2022] DELETE /identity/v3/users/08384d4a71da47f380bd748acd913e27 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 05:40:02.033152 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6d8f0a-6f6b-49bb-99e7-93bd57820b9f None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:02.033643 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6d8f0a-6f6b-49bb-99e7-93bd57820b9f None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:02.033742 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6d8f0a-6f6b-49bb-99e7-93bd57820b9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:02.051865 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-be6d8f0a-6f6b-49bb-99e7-93bd57820b9f None None] MFA Rules not processed for user `1eae540242d94c10b0ad33960a855f08`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:40:02.053126 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-be6d8f0a-6f6b-49bb-99e7-93bd57820b9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:40:02.101947 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:40:02.113486 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1192/2380] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 25 05:40:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:40:02.123165 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b5684b-fc06-441c-8684-22ca5489f310 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:02.124557 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-98b5684b-fc06-441c-8684-22ca5489f310 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:40:02.130997 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:40:02.185526 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b5684b-fc06-441c-8684-22ca5489f310 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:02.189188 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57929083-1060-4a8b-8810-b941510d3bb6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b2931200f99e43d79a50a9f6e02738e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92afba266cce4d1bad4134c351e74be3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:02.189842 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57929083-1060-4a8b-8810-b941510d3bb6 service neutron] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:02.190130 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57929083-1060-4a8b-8810-b941510d3bb6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:02.190410 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57929083-1060-4a8b-8810-b941510d3bb6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:02.200117 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57929083-1060-4a8b-8810-b941510d3bb6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:40:02.201604 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57929083-1060-4a8b-8810-b941510d3bb6 service neutron] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:40:02.211816 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1189/2381] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jul 25 05:40:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 05:40:02.384862 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be6d8f0a-6f6b-49bb-99e7-93bd57820b9f None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:02.386293 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-be6d8f0a-6f6b-49bb-99e7-93bd57820b9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:40:02.398875 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be6d8f0a-6f6b-49bb-99e7-93bd57820b9f None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:02.403506 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d5cf41c-c4a8-4c96-9950-960d60e8f640 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:02.404366 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5cf41c-c4a8-4c96-9950-960d60e8f640 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:02.404653 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5cf41c-c4a8-4c96-9950-960d60e8f640 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:02.404983 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5cf41c-c4a8-4c96-9950-960d60e8f640 admin admin] PATH_INFO: `/v3/projects/5d8e26c74f99499fb3890a0613364236` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:02.406181 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d5cf41c-c4a8-4c96-9950-960d60e8f640 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5d8e26c74f99499fb3890a0613364236)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:40:02.408089 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d5cf41c-c4a8-4c96-9950-960d60e8f640 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:40:02.456906 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-1d5cf41c-c4a8-4c96-9950-960d60e8f640 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5d8e26c74f99499fb3890a0613364236', 'request_id': 'req-1d5cf41c-c4a8-4c96-9950-960d60e8f640'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:40:02.462933 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-1d5cf41c-c4a8-4c96-9950-960d60e8f640 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5d8e26c74f99499fb3890a0613364236', 'request_id': 'req-1d5cf41c-c4a8-4c96-9950-960d60e8f640'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:40:02.471671 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-1d5cf41c-c4a8-4c96-9950-960d60e8f640 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5d8e26c74f99499fb3890a0613364236', 'request_id': 'req-1d5cf41c-c4a8-4c96-9950-960d60e8f640'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:40:02.485246 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-1d5cf41c-c4a8-4c96-9950-960d60e8f640 admin admin] The token cache is being invalidate because project 5d8e26c74f99499fb3890a0613364236 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80248) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 05:40:02.485533 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-1d5cf41c-c4a8-4c96-9950-960d60e8f640 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:40:02.486153 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-1d5cf41c-c4a8-4c96-9950-960d60e8f640 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:40:02.487801 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1193/2382] 10.222.0.33 () {58 vars in 1232 bytes} [Mon Jul 25 05:40:02 2022] DELETE /identity/v3/projects/5d8e26c74f99499fb3890a0613364236 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 25 05:40:02.493507 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ffa738c-da40-4de3-85ba-5124abb12aed None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:02.493802 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ffa738c-da40-4de3-85ba-5124abb12aed None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:02.494087 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ffa738c-da40-4de3-85ba-5124abb12aed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:02.522364 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-9ffa738c-da40-4de3-85ba-5124abb12aed None None] MFA Rules not processed for user `1eae540242d94c10b0ad33960a855f08`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:40:02.523782 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-9ffa738c-da40-4de3-85ba-5124abb12aed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:40:02.583609 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1190/2383] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 25 05:40:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 05:40:02.592036 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836e14ed-3f43-4407-aba8-e4b9ff809bd7 None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:02.592334 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836e14ed-3f43-4407-aba8-e4b9ff809bd7 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:02.592659 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836e14ed-3f43-4407-aba8-e4b9ff809bd7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:02.610796 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-836e14ed-3f43-4407-aba8-e4b9ff809bd7 None None] MFA Rules not processed for user `1eae540242d94c10b0ad33960a855f08`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:40:02.612009 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-836e14ed-3f43-4407-aba8-e4b9ff809bd7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:40:02.620672 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1194/2384] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 25 05:40:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:40:02.625059 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ffa738c-da40-4de3-85ba-5124abb12aed None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:02.635596 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ffa738c-da40-4de3-85ba-5124abb12aed None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:02.640289 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef6e546a-e43c-478a-bf25-30ad0e878035 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:02.640984 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6e546a-e43c-478a-bf25-30ad0e878035 admin admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:02.641384 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6e546a-e43c-478a-bf25-30ad0e878035 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:02.641676 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6e546a-e43c-478a-bf25-30ad0e878035 admin admin] PATH_INFO: `/v3/domains` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:02.642442 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef6e546a-e43c-478a-bf25-30ad0e878035 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:40:02.642754 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef6e546a-e43c-478a-bf25-30ad0e878035 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:40:02.644450 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef6e546a-e43c-478a-bf25-30ad0e878035 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:40:02.658771 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1191/2385] 10.222.0.33 () {58 vars in 1152 bytes} [Mon Jul 25 05:40:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 05:40:02.663760 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-836e14ed-3f43-4407-aba8-e4b9ff809bd7 None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:02.673080 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-836e14ed-3f43-4407-aba8-e4b9ff809bd7 None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:02.677881 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f71fc6-cb8a-48b1-bb07-6871be89ef52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:02.679579 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f71fc6-cb8a-48b1-bb07-6871be89ef52 admin admin] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:02.679893 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f71fc6-cb8a-48b1-bb07-6871be89ef52 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:02.680169 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f71fc6-cb8a-48b1-bb07-6871be89ef52 admin admin] PATH_INFO: `/v3/projects` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:02.680914 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5f71fc6-cb8a-48b1-bb07-6871be89ef52 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:40:02.682839 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5f71fc6-cb8a-48b1-bb07-6871be89ef52 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:40:02.708152 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1195/2386] 10.222.0.33 () {60 vars in 1152 bytes} [Mon Jul 25 05:40:02 2022] POST /identity/v3/projects => generated 389 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 05:40:02.713326 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef6e546a-e43c-478a-bf25-30ad0e878035 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:02.723229 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef6e546a-e43c-478a-bf25-30ad0e878035 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:02.782612 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee81ca3-68ca-473e-b037-5b9fb4abc17f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:02.783295 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee81ca3-68ca-473e-b037-5b9fb4abc17f admin admin] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:02.783583 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee81ca3-68ca-473e-b037-5b9fb4abc17f admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:02.783865 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee81ca3-68ca-473e-b037-5b9fb4abc17f admin admin] PATH_INFO: `/v3/users` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:02.784609 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bee81ca3-68ca-473e-b037-5b9fb4abc17f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:40:02.786660 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bee81ca3-68ca-473e-b037-5b9fb4abc17f admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:40:02.816967 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1192/2387] 10.222.0.33 () {60 vars in 1143 bytes} [Mon Jul 25 05:40:02 2022] POST /identity/v3/users => generated 295 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 05:40:02.818399 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f71fc6-cb8a-48b1-bb07-6871be89ef52 admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:02.830237 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f71fc6-cb8a-48b1-bb07-6871be89ef52 admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:02.836653 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52683008-d07d-4a6a-a5d2-ecbffa626e42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:02.837434 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52683008-d07d-4a6a-a5d2-ecbffa626e42 admin admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:02.837716 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52683008-d07d-4a6a-a5d2-ecbffa626e42 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:02.837993 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52683008-d07d-4a6a-a5d2-ecbffa626e42 admin admin] PATH_INFO: `/v3/roles` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:02.838734 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52683008-d07d-4a6a-a5d2-ecbffa626e42 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:40:02.840354 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52683008-d07d-4a6a-a5d2-ecbffa626e42 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:40:02.852035 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1196/2388] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jul 25 05:40:02 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 05:40:02.856077 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee81ca3-68ca-473e-b037-5b9fb4abc17f admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:02.867714 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee81ca3-68ca-473e-b037-5b9fb4abc17f admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:02.873431 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-029cacb9-b48b-4915-9d45-183c9081c611 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:02.874111 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029cacb9-b48b-4915-9d45-183c9081c611 admin admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:02.874401 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029cacb9-b48b-4915-9d45-183c9081c611 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:02.874683 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029cacb9-b48b-4915-9d45-183c9081c611 admin admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:02.875428 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-029cacb9-b48b-4915-9d45-183c9081c611 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:40:02.877079 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-029cacb9-b48b-4915-9d45-183c9081c611 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:40:02.887795 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1193/2389] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jul 25 05:40:02 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 25 05:40:02.891980 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52683008-d07d-4a6a-a5d2-ecbffa626e42 admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:02.904162 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52683008-d07d-4a6a-a5d2-ecbffa626e42 admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:02.910398 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4231e15b-3fef-4071-a8fc-cc599af632fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:02.911359 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4231e15b-3fef-4071-a8fc-cc599af632fe admin admin] REQUEST_METHOD: `PUT` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:02.911717 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4231e15b-3fef-4071-a8fc-cc599af632fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:02.912069 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4231e15b-3fef-4071-a8fc-cc599af632fe admin admin] PATH_INFO: `/v3/projects/e919d8a898674f11ac789bfd48f0a3ee/users/419dc8776eb64eb7a54ab96474d6595d/roles/5074e74e35f54d4b9bb415c34999a82a` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:02.927495 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4231e15b-3fef-4071-a8fc-cc599af632fe admin admin] RBAC: Authorizing `identity:create_grant(project_id=e919d8a898674f11ac789bfd48f0a3ee, user_id=419dc8776eb64eb7a54ab96474d6595d, role_id=5074e74e35f54d4b9bb415c34999a82a)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:40:02.930738 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4231e15b-3fef-4071-a8fc-cc599af632fe admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:40:02.943311 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1197/2390] 10.222.0.33 () {60 vars in 1482 bytes} [Mon Jul 25 05:40:02 2022] PUT /identity/v3/projects/e919d8a898674f11ac789bfd48f0a3ee/users/419dc8776eb64eb7a54ab96474d6595d/roles/5074e74e35f54d4b9bb415c34999a82a => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 05:40:02.954895 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a64fe93-67a1-47ca-b9c9-41436a605033 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:02.955722 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a64fe93-67a1-47ca-b9c9-41436a605033 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:02.955722 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a64fe93-67a1-47ca-b9c9-41436a605033 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:02.980805 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:40:02.984193 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-4a64fe93-67a1-47ca-b9c9-41436a605033 None None] MFA Rules not processed for user `419dc8776eb64eb7a54ab96474d6595d`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:40:02.986014 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-4a64fe93-67a1-47ca-b9c9-41436a605033 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:40:03.051001 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1194/2391] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 25 05:40:02 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:40:03.061898 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4231e15b-3fef-4071-a8fc-cc599af632fe admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:03.063640 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-4231e15b-3fef-4071-a8fc-cc599af632fe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:40:03.121403 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4231e15b-3fef-4071-a8fc-cc599af632fe admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:03.125220 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd77f18-05a6-4e3c-9826-6aa3d5d80d8e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db57b478c25643dbaf936f90f6e17d0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92afba266cce4d1bad4134c351e74be3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:03.125873 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd77f18-05a6-4e3c-9826-6aa3d5d80d8e service cinder] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:03.126161 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd77f18-05a6-4e3c-9826-6aa3d5d80d8e service cinder] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:03.126445 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd77f18-05a6-4e3c-9826-6aa3d5d80d8e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:03.135627 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bd77f18-05a6-4e3c-9826-6aa3d5d80d8e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:40:03.137174 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bd77f18-05a6-4e3c-9826-6aa3d5d80d8e service cinder] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:40:03.146013 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1198/2392] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jul 25 05:40:03 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 05:40:13.047843 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=26 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:40:13.141967 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=82,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:40:22.031730 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a64fe93-67a1-47ca-b9c9-41436a605033 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:22.033351 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-4a64fe93-67a1-47ca-b9c9-41436a605033 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:40:22.103039 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a64fe93-67a1-47ca-b9c9-41436a605033 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:22.108674 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de6a75af-dbf0-4c5e-aa6e-8e29eff52df8 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '317790016e7047fe96af73925de5d872', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92afba266cce4d1bad4134c351e74be3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:22.108674 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6a75af-dbf0-4c5e-aa6e-8e29eff52df8 service placement] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:22.108992 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6a75af-dbf0-4c5e-aa6e-8e29eff52df8 service placement] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:22.109270 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6a75af-dbf0-4c5e-aa6e-8e29eff52df8 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:22.111057 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-de6a75af-dbf0-4c5e-aa6e-8e29eff52df8 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:40:22.123100 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de6a75af-dbf0-4c5e-aa6e-8e29eff52df8 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:40:22.125103 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de6a75af-dbf0-4c5e-aa6e-8e29eff52df8 service placement] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:40:22.135732 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1195/2393] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jul 25 05:40:22 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 05:40:25.402673 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd77f18-05a6-4e3c-9826-6aa3d5d80d8e service cinder] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:25.404146 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-8bd77f18-05a6-4e3c-9826-6aa3d5d80d8e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:40:25.463677 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd77f18-05a6-4e3c-9826-6aa3d5d80d8e service cinder] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:25.467310 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0bf4873-16f7-4ab2-8eb5-6a73e7820c0e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b2931200f99e43d79a50a9f6e02738e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92afba266cce4d1bad4134c351e74be3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:25.467957 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0bf4873-16f7-4ab2-8eb5-6a73e7820c0e service neutron] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:25.468245 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0bf4873-16f7-4ab2-8eb5-6a73e7820c0e service neutron] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:25.468527 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0bf4873-16f7-4ab2-8eb5-6a73e7820c0e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:25.470330 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-c0bf4873-16f7-4ab2-8eb5-6a73e7820c0e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:40:25.535535 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0bf4873-16f7-4ab2-8eb5-6a73e7820c0e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:40:25.537091 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0bf4873-16f7-4ab2-8eb5-6a73e7820c0e service neutron] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:40:25.547079 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1199/2394] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jul 25 05:40:25 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 05:40:26.659379 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de6a75af-dbf0-4c5e-aa6e-8e29eff52df8 service placement] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:26.723857 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de6a75af-dbf0-4c5e-aa6e-8e29eff52df8 service placement] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:26.729241 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a34e7be-30af-4f76-990d-4f3648cfa3ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:26.730117 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a34e7be-30af-4f76-990d-4f3648cfa3ad admin admin] REQUEST_METHOD: `DELETE` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:26.730401 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a34e7be-30af-4f76-990d-4f3648cfa3ad admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:26.730673 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a34e7be-30af-4f76-990d-4f3648cfa3ad admin admin] PATH_INFO: `/v3/users/419dc8776eb64eb7a54ab96474d6595d` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:26.731896 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a34e7be-30af-4f76-990d-4f3648cfa3ad admin admin] RBAC: Authorizing `identity:delete_user(user_id=419dc8776eb64eb7a54ab96474d6595d)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:40:26.733872 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a34e7be-30af-4f76-990d-4f3648cfa3ad admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:40:26.783534 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-2a34e7be-30af-4f76-990d-4f3648cfa3ad admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '419dc8776eb64eb7a54ab96474d6595d'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:40:26.784407 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-2a34e7be-30af-4f76-990d-4f3648cfa3ad admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '419dc8776eb64eb7a54ab96474d6595d'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:40:26.792149 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-2a34e7be-30af-4f76-990d-4f3648cfa3ad admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '419dc8776eb64eb7a54ab96474d6595d'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:40:26.804655 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-2a34e7be-30af-4f76-990d-4f3648cfa3ad admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '419dc8776eb64eb7a54ab96474d6595d'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:40:26.806479 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-2a34e7be-30af-4f76-990d-4f3648cfa3ad admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '419dc8776eb64eb7a54ab96474d6595d'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:40:26.820913 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1196/2395] 10.222.0.33 () {58 vars in 1223 bytes} [Mon Jul 25 05:40:26 2022] DELETE /identity/v3/users/419dc8776eb64eb7a54ab96474d6595d => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 05:40:26.825478 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4026986b-b7cf-40f5-b257-c0b25d667027 None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:26.825961 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4026986b-b7cf-40f5-b257-c0b25d667027 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:26.826076 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4026986b-b7cf-40f5-b257-c0b25d667027 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:26.844009 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-4026986b-b7cf-40f5-b257-c0b25d667027 None None] MFA Rules not processed for user `1eae540242d94c10b0ad33960a855f08`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:40:26.845268 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-4026986b-b7cf-40f5-b257-c0b25d667027 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:40:26.909743 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1200/2396] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 25 05:40:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:40:26.919580 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a34e7be-30af-4f76-990d-4f3648cfa3ad admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:26.921145 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-2a34e7be-30af-4f76-990d-4f3648cfa3ad admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:40:26.933001 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:40:26.985525 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a34e7be-30af-4f76-990d-4f3648cfa3ad admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:26.989226 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-205f9f3a-647c-4e6d-b42f-eda8b1a79d75 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b2931200f99e43d79a50a9f6e02738e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92afba266cce4d1bad4134c351e74be3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:26.989883 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205f9f3a-647c-4e6d-b42f-eda8b1a79d75 service neutron] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:26.990178 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205f9f3a-647c-4e6d-b42f-eda8b1a79d75 service neutron] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:26.990455 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205f9f3a-647c-4e6d-b42f-eda8b1a79d75 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:27.000166 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-205f9f3a-647c-4e6d-b42f-eda8b1a79d75 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:40:27.001647 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-205f9f3a-647c-4e6d-b42f-eda8b1a79d75 service neutron] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:40:27.012208 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1197/2397] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jul 25 05:40:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 25 05:40:27.313423 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4026986b-b7cf-40f5-b257-c0b25d667027 None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:27.314973 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-4026986b-b7cf-40f5-b257-c0b25d667027 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:40:27.347692 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4026986b-b7cf-40f5-b257-c0b25d667027 None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:27.354245 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b99e9fd4-8eab-471a-a40d-6042f9aeb6d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:27.356708 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99e9fd4-8eab-471a-a40d-6042f9aeb6d9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:27.357039 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99e9fd4-8eab-471a-a40d-6042f9aeb6d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:27.357318 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99e9fd4-8eab-471a-a40d-6042f9aeb6d9 admin admin] PATH_INFO: `/v3/projects/e919d8a898674f11ac789bfd48f0a3ee` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:27.358598 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b99e9fd4-8eab-471a-a40d-6042f9aeb6d9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e919d8a898674f11ac789bfd48f0a3ee)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:40:27.374626 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b99e9fd4-8eab-471a-a40d-6042f9aeb6d9 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:40:27.453397 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:40:27.456727 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-b99e9fd4-8eab-471a-a40d-6042f9aeb6d9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e919d8a898674f11ac789bfd48f0a3ee', 'request_id': 'req-b99e9fd4-8eab-471a-a40d-6042f9aeb6d9'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:40:27.464209 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-b99e9fd4-8eab-471a-a40d-6042f9aeb6d9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e919d8a898674f11ac789bfd48f0a3ee', 'request_id': 'req-b99e9fd4-8eab-471a-a40d-6042f9aeb6d9'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:40:27.472119 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-b99e9fd4-8eab-471a-a40d-6042f9aeb6d9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e919d8a898674f11ac789bfd48f0a3ee', 'request_id': 'req-b99e9fd4-8eab-471a-a40d-6042f9aeb6d9'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:40:27.488527 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-b99e9fd4-8eab-471a-a40d-6042f9aeb6d9 admin admin] The token cache is being invalidate because project e919d8a898674f11ac789bfd48f0a3ee was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80248) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 05:40:27.488902 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-b99e9fd4-8eab-471a-a40d-6042f9aeb6d9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:40:27.489524 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-b99e9fd4-8eab-471a-a40d-6042f9aeb6d9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:40:27.496138 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1201/2398] 10.222.0.33 () {58 vars in 1232 bytes} [Mon Jul 25 05:40:27 2022] DELETE /identity/v3/projects/e919d8a898674f11ac789bfd48f0a3ee => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 05:40:27.498312 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1519989d-3764-4cca-be90-212961efe01d None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:27.498608 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1519989d-3764-4cca-be90-212961efe01d None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:27.498891 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1519989d-3764-4cca-be90-212961efe01d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:27.518508 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-1519989d-3764-4cca-be90-212961efe01d None None] MFA Rules not processed for user `1eae540242d94c10b0ad33960a855f08`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:40:27.519754 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-1519989d-3764-4cca-be90-212961efe01d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:40:27.597067 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1198/2399] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 25 05:40:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:40:27.599244 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec75e5d-e029-45e3-9b93-7bf54fe9b008 None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:27.599862 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec75e5d-e029-45e3-9b93-7bf54fe9b008 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:27.600154 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec75e5d-e029-45e3-9b93-7bf54fe9b008 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:27.623421 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-7ec75e5d-e029-45e3-9b93-7bf54fe9b008 None None] MFA Rules not processed for user `1eae540242d94c10b0ad33960a855f08`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:40:27.624647 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-7ec75e5d-e029-45e3-9b93-7bf54fe9b008 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:40:27.632659 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1202/2400] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 25 05:40:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:40:27.639003 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1519989d-3764-4cca-be90-212961efe01d None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:27.649033 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1519989d-3764-4cca-be90-212961efe01d None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:27.656522 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec61e45-0b3b-4eca-bd5c-3217bb4cb8ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:27.657738 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec61e45-0b3b-4eca-bd5c-3217bb4cb8ed admin admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:27.658030 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec61e45-0b3b-4eca-bd5c-3217bb4cb8ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:27.658339 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec61e45-0b3b-4eca-bd5c-3217bb4cb8ed admin admin] PATH_INFO: `/v3/domains` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:27.659093 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ec61e45-0b3b-4eca-bd5c-3217bb4cb8ed admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:40:27.659400 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ec61e45-0b3b-4eca-bd5c-3217bb4cb8ed admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:40:27.661051 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ec61e45-0b3b-4eca-bd5c-3217bb4cb8ed admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:40:27.675059 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1199/2401] 10.222.0.33 () {58 vars in 1152 bytes} [Mon Jul 25 05:40:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 05:40:27.679548 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec75e5d-e029-45e3-9b93-7bf54fe9b008 None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:27.691522 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec75e5d-e029-45e3-9b93-7bf54fe9b008 None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:27.698381 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9722dbae-930c-4dde-9ac6-5dd21733a606 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:27.699059 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9722dbae-930c-4dde-9ac6-5dd21733a606 admin admin] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:27.699343 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9722dbae-930c-4dde-9ac6-5dd21733a606 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:27.699765 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9722dbae-930c-4dde-9ac6-5dd21733a606 admin admin] PATH_INFO: `/v3/projects` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:27.701484 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9722dbae-930c-4dde-9ac6-5dd21733a606 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:40:27.703803 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9722dbae-930c-4dde-9ac6-5dd21733a606 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:40:27.747690 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1203/2402] 10.222.0.33 () {60 vars in 1152 bytes} [Mon Jul 25 05:40:27 2022] POST /identity/v3/projects => generated 399 bytes in 65 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 25 05:40:27.748962 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec61e45-0b3b-4eca-bd5c-3217bb4cb8ed admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:27.761224 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec61e45-0b3b-4eca-bd5c-3217bb4cb8ed admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:27.825947 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b29edc21-a9b5-420d-9752-08d6b577b3aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:27.827198 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29edc21-a9b5-420d-9752-08d6b577b3aa admin admin] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:27.827497 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29edc21-a9b5-420d-9752-08d6b577b3aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:27.827783 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29edc21-a9b5-420d-9752-08d6b577b3aa admin admin] PATH_INFO: `/v3/users` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:27.828567 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b29edc21-a9b5-420d-9752-08d6b577b3aa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:40:27.830972 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b29edc21-a9b5-420d-9752-08d6b577b3aa admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:40:27.857666 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1200/2403] 10.222.0.33 () {60 vars in 1143 bytes} [Mon Jul 25 05:40:27 2022] POST /identity/v3/users => generated 300 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 25 05:40:27.861640 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9722dbae-930c-4dde-9ac6-5dd21733a606 admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:27.875530 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9722dbae-930c-4dde-9ac6-5dd21733a606 admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:27.881055 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c8b8fcc-86bf-4513-85ed-e6983a8beb05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:27.881740 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8b8fcc-86bf-4513-85ed-e6983a8beb05 admin admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:27.882035 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8b8fcc-86bf-4513-85ed-e6983a8beb05 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:27.882313 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8b8fcc-86bf-4513-85ed-e6983a8beb05 admin admin] PATH_INFO: `/v3/roles` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:27.883043 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c8b8fcc-86bf-4513-85ed-e6983a8beb05 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:40:27.885494 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c8b8fcc-86bf-4513-85ed-e6983a8beb05 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:40:27.896634 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1204/2404] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jul 25 05:40:27 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 25 05:40:27.901051 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b29edc21-a9b5-420d-9752-08d6b577b3aa admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:27.925700 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b29edc21-a9b5-420d-9752-08d6b577b3aa admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:27.931848 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e99f204-4802-4be8-a8d0-8eadd36099b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:27.932816 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e99f204-4802-4be8-a8d0-8eadd36099b7 admin admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:27.933340 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e99f204-4802-4be8-a8d0-8eadd36099b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:27.933833 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e99f204-4802-4be8-a8d0-8eadd36099b7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:27.934586 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e99f204-4802-4be8-a8d0-8eadd36099b7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:40:27.936272 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e99f204-4802-4be8-a8d0-8eadd36099b7 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:40:27.950836 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1201/2405] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jul 25 05:40:27 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 25 05:40:27.952167 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c8b8fcc-86bf-4513-85ed-e6983a8beb05 admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:27.963841 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c8b8fcc-86bf-4513-85ed-e6983a8beb05 admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:27.969465 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0537514-bbbf-4316-8e38-648a3e6a1b17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:27.970806 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0537514-bbbf-4316-8e38-648a3e6a1b17 admin admin] REQUEST_METHOD: `PUT` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:27.971194 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0537514-bbbf-4316-8e38-648a3e6a1b17 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:27.971568 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0537514-bbbf-4316-8e38-648a3e6a1b17 admin admin] PATH_INFO: `/v3/projects/6edbde986d534e60bb09e994c9e5838d/users/3b861c3340ca42859dc99833540c064a/roles/5074e74e35f54d4b9bb415c34999a82a` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:27.990350 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0537514-bbbf-4316-8e38-648a3e6a1b17 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6edbde986d534e60bb09e994c9e5838d, user_id=3b861c3340ca42859dc99833540c064a, role_id=5074e74e35f54d4b9bb415c34999a82a)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:40:27.993689 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0537514-bbbf-4316-8e38-648a3e6a1b17 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:40:28.016865 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1205/2406] 10.222.0.33 () {60 vars in 1482 bytes} [Mon Jul 25 05:40:27 2022] PUT /identity/v3/projects/6edbde986d534e60bb09e994c9e5838d/users/3b861c3340ca42859dc99833540c064a/roles/5074e74e35f54d4b9bb415c34999a82a => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 05:40:28.023020 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7ab2c9-58fb-4052-a4de-98fee3f3f8fb None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:28.023443 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7ab2c9-58fb-4052-a4de-98fee3f3f8fb None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:28.023767 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7ab2c9-58fb-4052-a4de-98fee3f3f8fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:28.055077 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-bf7ab2c9-58fb-4052-a4de-98fee3f3f8fb None None] MFA Rules not processed for user `3b861c3340ca42859dc99833540c064a`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:40:28.057121 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-bf7ab2c9-58fb-4052-a4de-98fee3f3f8fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:40:28.062954 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:40:28.132550 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1202/2407] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 25 05:40:28 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 05:40:28.145509 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0537514-bbbf-4316-8e38-648a3e6a1b17 admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:28.147199 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-d0537514-bbbf-4316-8e38-648a3e6a1b17 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:40:28.210247 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0537514-bbbf-4316-8e38-648a3e6a1b17 admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:28.215229 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4950e36-1322-44d7-bdcc-b58f2af07a1a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db57b478c25643dbaf936f90f6e17d0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92afba266cce4d1bad4134c351e74be3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:28.215882 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4950e36-1322-44d7-bdcc-b58f2af07a1a service cinder] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:28.216167 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4950e36-1322-44d7-bdcc-b58f2af07a1a service cinder] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:28.216444 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4950e36-1322-44d7-bdcc-b58f2af07a1a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:28.224972 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4950e36-1322-44d7-bdcc-b58f2af07a1a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:40:28.226414 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4950e36-1322-44d7-bdcc-b58f2af07a1a service cinder] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:40:28.235809 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1206/2408] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jul 25 05:40:28 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 05:40:38.129422 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=24 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:40:38.231745 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:DELETE=4,keystone:SELECT=60,keystone:INSERT=3 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:40:39.492383 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf7ab2c9-58fb-4052-a4de-98fee3f3f8fb None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:39.560639 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf7ab2c9-58fb-4052-a4de-98fee3f3f8fb None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:39.565241 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84874abb-2a47-42ce-9666-0eef767109d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:39.566129 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84874abb-2a47-42ce-9666-0eef767109d2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:39.566415 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84874abb-2a47-42ce-9666-0eef767109d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:39.566703 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84874abb-2a47-42ce-9666-0eef767109d2 admin admin] PATH_INFO: `/v3/users/3b861c3340ca42859dc99833540c064a` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:39.568032 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84874abb-2a47-42ce-9666-0eef767109d2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3b861c3340ca42859dc99833540c064a)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:40:39.570018 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84874abb-2a47-42ce-9666-0eef767109d2 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:40:39.618093 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-84874abb-2a47-42ce-9666-0eef767109d2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3b861c3340ca42859dc99833540c064a'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:40:39.619318 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-84874abb-2a47-42ce-9666-0eef767109d2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3b861c3340ca42859dc99833540c064a'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:40:39.626291 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-84874abb-2a47-42ce-9666-0eef767109d2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3b861c3340ca42859dc99833540c064a'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:40:39.638901 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-84874abb-2a47-42ce-9666-0eef767109d2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3b861c3340ca42859dc99833540c064a'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:40:39.639546 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-84874abb-2a47-42ce-9666-0eef767109d2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3b861c3340ca42859dc99833540c064a'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:40:39.652838 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1203/2409] 10.222.0.33 () {58 vars in 1223 bytes} [Mon Jul 25 05:40:39 2022] DELETE /identity/v3/users/3b861c3340ca42859dc99833540c064a => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 05:40:39.657564 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66154946-7082-4756-b817-3910f24aff7d None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:39.658069 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66154946-7082-4756-b817-3910f24aff7d None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:39.658166 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66154946-7082-4756-b817-3910f24aff7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:39.675002 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-66154946-7082-4756-b817-3910f24aff7d None None] MFA Rules not processed for user `1eae540242d94c10b0ad33960a855f08`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:40:39.676212 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-66154946-7082-4756-b817-3910f24aff7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:40:39.736927 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1207/2410] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 25 05:40:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:40:39.747199 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84874abb-2a47-42ce-9666-0eef767109d2 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:39.748625 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-84874abb-2a47-42ce-9666-0eef767109d2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:40:39.809337 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84874abb-2a47-42ce-9666-0eef767109d2 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:39.813167 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc09db6e-cdfc-46e2-90bc-d959c8436bd3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b2931200f99e43d79a50a9f6e02738e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92afba266cce4d1bad4134c351e74be3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:39.813828 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc09db6e-cdfc-46e2-90bc-d959c8436bd3 service neutron] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:39.814116 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc09db6e-cdfc-46e2-90bc-d959c8436bd3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:39.814396 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc09db6e-cdfc-46e2-90bc-d959c8436bd3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:39.824329 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc09db6e-cdfc-46e2-90bc-d959c8436bd3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:40:39.825820 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc09db6e-cdfc-46e2-90bc-d959c8436bd3 service neutron] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:40:39.835200 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1204/2411] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jul 25 05:40:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 05:40:40.099649 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66154946-7082-4756-b817-3910f24aff7d None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:40.101117 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-66154946-7082-4756-b817-3910f24aff7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:40:40.113559 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66154946-7082-4756-b817-3910f24aff7d None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:40.118321 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d9e0f8-d51e-4683-8deb-9cdf87b4dd0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:40.119901 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d9e0f8-d51e-4683-8deb-9cdf87b4dd0b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:40.120228 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d9e0f8-d51e-4683-8deb-9cdf87b4dd0b admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:40.120505 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d9e0f8-d51e-4683-8deb-9cdf87b4dd0b admin admin] PATH_INFO: `/v3/projects/6edbde986d534e60bb09e994c9e5838d` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:40.121737 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62d9e0f8-d51e-4683-8deb-9cdf87b4dd0b admin admin] RBAC: Authorizing `identity:delete_project(project_id=6edbde986d534e60bb09e994c9e5838d)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:40:40.123656 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62d9e0f8-d51e-4683-8deb-9cdf87b4dd0b admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:40:40.170971 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-62d9e0f8-d51e-4683-8deb-9cdf87b4dd0b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6edbde986d534e60bb09e994c9e5838d', 'request_id': 'req-62d9e0f8-d51e-4683-8deb-9cdf87b4dd0b'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:40:40.176723 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-62d9e0f8-d51e-4683-8deb-9cdf87b4dd0b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6edbde986d534e60bb09e994c9e5838d', 'request_id': 'req-62d9e0f8-d51e-4683-8deb-9cdf87b4dd0b'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:40:40.183686 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-62d9e0f8-d51e-4683-8deb-9cdf87b4dd0b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6edbde986d534e60bb09e994c9e5838d', 'request_id': 'req-62d9e0f8-d51e-4683-8deb-9cdf87b4dd0b'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:40:40.195710 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-62d9e0f8-d51e-4683-8deb-9cdf87b4dd0b admin admin] The token cache is being invalidate because project 6edbde986d534e60bb09e994c9e5838d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80248) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 05:40:40.195998 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-62d9e0f8-d51e-4683-8deb-9cdf87b4dd0b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:40:40.196615 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-62d9e0f8-d51e-4683-8deb-9cdf87b4dd0b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:40:40.199095 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1208/2412] 10.222.0.33 () {58 vars in 1232 bytes} [Mon Jul 25 05:40:40 2022] DELETE /identity/v3/projects/6edbde986d534e60bb09e994c9e5838d => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 05:40:49.831150 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=79,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:40:50.192989 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=54,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:41:04.472756 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0900b2d9-ee1f-4d0f-a1f1-0e0206fc520d None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:04.473286 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0900b2d9-ee1f-4d0f-a1f1-0e0206fc520d None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:04.473491 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0900b2d9-ee1f-4d0f-a1f1-0e0206fc520d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:04.491864 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-0900b2d9-ee1f-4d0f-a1f1-0e0206fc520d None None] MFA Rules not processed for user `69f3276517de4f889dddd3180436ede9`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:41:04.493117 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-0900b2d9-ee1f-4d0f-a1f1-0e0206fc520d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:41:04.550291 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1205/2413] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 25 05:41:04 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:41:10.569564 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3952af36-cdbf-45bb-9218-6fb13da3f1c2 None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:10.570110 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3952af36-cdbf-45bb-9218-6fb13da3f1c2 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:10.570110 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3952af36-cdbf-45bb-9218-6fb13da3f1c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:10.588457 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-3952af36-cdbf-45bb-9218-6fb13da3f1c2 None None] MFA Rules not processed for user `ce0944d760224239bd16aa0981db0eec`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:41:10.589728 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-3952af36-cdbf-45bb-9218-6fb13da3f1c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:41:10.647874 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1209/2414] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 25 05:41:10 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:41:14.543484 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=25 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:41:20.641032 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=25 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:41:24.907075 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9fd7e7-c7ad-4b33-9a43-cb228d44ec77 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:24.907617 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9fd7e7-c7ad-4b33-9a43-cb228d44ec77 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:24.907790 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9fd7e7-c7ad-4b33-9a43-cb228d44ec77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:24.925771 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-4b9fd7e7-c7ad-4b33-9a43-cb228d44ec77 None None] MFA Rules not processed for user `37c1bb8500f74013b073febfc0d96217`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:41:24.927053 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-4b9fd7e7-c7ad-4b33-9a43-cb228d44ec77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:41:24.983151 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1206/2415] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 25 05:41:24 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:41:34.976366 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=25 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:42:06.143320 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3952af36-cdbf-45bb-9218-6fb13da3f1c2 None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:42:06.144729 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-3952af36-cdbf-45bb-9218-6fb13da3f1c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:42:06.205057 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3952af36-cdbf-45bb-9218-6fb13da3f1c2 None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:42:06.208648 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f1b8cc-a322-4c66-915a-e8b1b4b5d540 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db57b478c25643dbaf936f90f6e17d0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92afba266cce4d1bad4134c351e74be3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:42:06.209323 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f1b8cc-a322-4c66-915a-e8b1b4b5d540 service cinder] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:42:06.209611 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f1b8cc-a322-4c66-915a-e8b1b4b5d540 service cinder] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:42:06.209891 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f1b8cc-a322-4c66-915a-e8b1b4b5d540 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:42:06.211657 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-b0f1b8cc-a322-4c66-915a-e8b1b4b5d540 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:42:06.288830 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0f1b8cc-a322-4c66-915a-e8b1b4b5d540 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:42:06.290269 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0f1b8cc-a322-4c66-915a-e8b1b4b5d540 service cinder] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:42:06.301791 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1210/2416] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jul 25 05:42:06 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 159 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 05:42:10.904857 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b9fd7e7-c7ad-4b33-9a43-cb228d44ec77 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:42:10.906309 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-4b9fd7e7-c7ad-4b33-9a43-cb228d44ec77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:42:10.919515 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b9fd7e7-c7ad-4b33-9a43-cb228d44ec77 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:42:10.923788 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bfb698c-2127-4c37-ae56-4d34bf9750b8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db57b478c25643dbaf936f90f6e17d0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92afba266cce4d1bad4134c351e74be3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:42:10.924954 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bfb698c-2127-4c37-ae56-4d34bf9750b8 service cinder] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:42:10.925470 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bfb698c-2127-4c37-ae56-4d34bf9750b8 service cinder] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:42:10.926025 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bfb698c-2127-4c37-ae56-4d34bf9750b8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:42:10.927884 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-2bfb698c-2127-4c37-ae56-4d34bf9750b8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:42:10.939159 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bfb698c-2127-4c37-ae56-4d34bf9750b8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:42:10.940577 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bfb698c-2127-4c37-ae56-4d34bf9750b8 service cinder] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:42:10.949569 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1207/2417] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jul 25 05:42:10 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 05:42:11.049985 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f1b8cc-a322-4c66-915a-e8b1b4b5d540 service cinder] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:42:11.058593 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f1b8cc-a322-4c66-915a-e8b1b4b5d540 service cinder] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:42:11.062153 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c373d2c-295f-48b2-aa5c-b4da52fda55e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db57b478c25643dbaf936f90f6e17d0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92afba266cce4d1bad4134c351e74be3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:42:11.063063 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c373d2c-295f-48b2-aa5c-b4da52fda55e service cinder] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:42:11.063353 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c373d2c-295f-48b2-aa5c-b4da52fda55e service cinder] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:42:11.063627 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c373d2c-295f-48b2-aa5c-b4da52fda55e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:42:11.065417 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-8c373d2c-295f-48b2-aa5c-b4da52fda55e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:42:11.079153 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c373d2c-295f-48b2-aa5c-b4da52fda55e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:42:11.080548 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c373d2c-295f-48b2-aa5c-b4da52fda55e service cinder] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:42:11.091503 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1211/2418] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jul 25 05:42:11 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 05:42:12.136908 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bfb698c-2127-4c37-ae56-4d34bf9750b8 service cinder] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:42:12.138484 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-2bfb698c-2127-4c37-ae56-4d34bf9750b8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:42:12.204847 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bfb698c-2127-4c37-ae56-4d34bf9750b8 service cinder] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:42:12.209348 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ba0147-8fff-406b-992f-28baf3a49c8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:42:12.210213 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ba0147-8fff-406b-992f-28baf3a49c8d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:42:12.210498 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ba0147-8fff-406b-992f-28baf3a49c8d admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:42:12.210775 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ba0147-8fff-406b-992f-28baf3a49c8d admin admin] PATH_INFO: `/v3/users/ce0944d760224239bd16aa0981db0eec` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:42:12.211955 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48ba0147-8fff-406b-992f-28baf3a49c8d admin admin] RBAC: Authorizing `identity:delete_user(user_id=ce0944d760224239bd16aa0981db0eec)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:42:12.213897 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48ba0147-8fff-406b-992f-28baf3a49c8d admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:42:12.258989 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-48ba0147-8fff-406b-992f-28baf3a49c8d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ce0944d760224239bd16aa0981db0eec'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:42:12.259654 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-48ba0147-8fff-406b-992f-28baf3a49c8d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ce0944d760224239bd16aa0981db0eec'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:42:12.267724 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-48ba0147-8fff-406b-992f-28baf3a49c8d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ce0944d760224239bd16aa0981db0eec'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:42:12.280271 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-48ba0147-8fff-406b-992f-28baf3a49c8d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ce0944d760224239bd16aa0981db0eec'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:42:12.281079 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-48ba0147-8fff-406b-992f-28baf3a49c8d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ce0944d760224239bd16aa0981db0eec'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:42:12.296401 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1208/2419] 10.222.0.33 () {58 vars in 1223 bytes} [Mon Jul 25 05:42:12 2022] DELETE /identity/v3/users/ce0944d760224239bd16aa0981db0eec => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 05:42:12.477678 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c373d2c-295f-48b2-aa5c-b4da52fda55e service cinder] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:42:12.489060 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c373d2c-295f-48b2-aa5c-b4da52fda55e service cinder] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:42:12.547987 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af7b0cb-43d6-43b6-84ee-c438350b2520 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:42:12.548919 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af7b0cb-43d6-43b6-84ee-c438350b2520 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:42:12.549218 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af7b0cb-43d6-43b6-84ee-c438350b2520 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:42:12.549505 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af7b0cb-43d6-43b6-84ee-c438350b2520 admin admin] PATH_INFO: `/v3/projects/213b3114e35d4016b3169364760edb17` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:42:12.550757 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5af7b0cb-43d6-43b6-84ee-c438350b2520 admin admin] RBAC: Authorizing `identity:delete_project(project_id=213b3114e35d4016b3169364760edb17)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:42:12.552726 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5af7b0cb-43d6-43b6-84ee-c438350b2520 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:42:12.587010 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:42:12.600763 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-5af7b0cb-43d6-43b6-84ee-c438350b2520 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '213b3114e35d4016b3169364760edb17', 'request_id': 'req-5af7b0cb-43d6-43b6-84ee-c438350b2520'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:42:12.606331 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-5af7b0cb-43d6-43b6-84ee-c438350b2520 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '213b3114e35d4016b3169364760edb17', 'request_id': 'req-5af7b0cb-43d6-43b6-84ee-c438350b2520'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:42:12.613151 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-5af7b0cb-43d6-43b6-84ee-c438350b2520 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '213b3114e35d4016b3169364760edb17', 'request_id': 'req-5af7b0cb-43d6-43b6-84ee-c438350b2520'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:42:12.626581 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-5af7b0cb-43d6-43b6-84ee-c438350b2520 admin admin] The token cache is being invalidate because project 213b3114e35d4016b3169364760edb17 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80248) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 05:42:12.626849 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-5af7b0cb-43d6-43b6-84ee-c438350b2520 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:42:12.627475 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-5af7b0cb-43d6-43b6-84ee-c438350b2520 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:42:12.629120 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1212/2420] 10.222.0.33 () {58 vars in 1232 bytes} [Mon Jul 25 05:42:12 2022] DELETE /identity/v3/projects/213b3114e35d4016b3169364760edb17 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 05:42:12.634081 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2826f385-f323-469f-bc8a-668b59a8bf6f None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:42:12.634377 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2826f385-f323-469f-bc8a-668b59a8bf6f None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:42:12.634660 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2826f385-f323-469f-bc8a-668b59a8bf6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:42:12.652680 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-2826f385-f323-469f-bc8a-668b59a8bf6f None None] MFA Rules not processed for user `1eae540242d94c10b0ad33960a855f08`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:42:12.653828 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-2826f385-f323-469f-bc8a-668b59a8bf6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:42:12.711790 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1209/2421] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 25 05:42:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:42:12.720082 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b74ceb-94b6-4e7c-9b6d-a97a8c7ed366 None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:42:12.720376 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b74ceb-94b6-4e7c-9b6d-a97a8c7ed366 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:42:12.720662 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b74ceb-94b6-4e7c-9b6d-a97a8c7ed366 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:42:12.739350 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-d7b74ceb-94b6-4e7c-9b6d-a97a8c7ed366 None None] MFA Rules not processed for user `1eae540242d94c10b0ad33960a855f08`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:42:12.740568 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-d7b74ceb-94b6-4e7c-9b6d-a97a8c7ed366 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:42:12.748524 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1213/2422] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 25 05:42:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:42:12.752730 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2826f385-f323-469f-bc8a-668b59a8bf6f None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:42:12.762483 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2826f385-f323-469f-bc8a-668b59a8bf6f None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:42:12.767591 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ebe331-7941-4c11-8ea0-8e9c4e4f3e02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:42:12.768246 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ebe331-7941-4c11-8ea0-8e9c4e4f3e02 admin admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:42:12.768535 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ebe331-7941-4c11-8ea0-8e9c4e4f3e02 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:42:12.768849 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ebe331-7941-4c11-8ea0-8e9c4e4f3e02 admin admin] PATH_INFO: `/v3/domains` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:42:12.769613 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28ebe331-7941-4c11-8ea0-8e9c4e4f3e02 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:42:12.769930 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28ebe331-7941-4c11-8ea0-8e9c4e4f3e02 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:42:12.771515 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28ebe331-7941-4c11-8ea0-8e9c4e4f3e02 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:42:12.773018 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:42:12.786259 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1210/2423] 10.222.0.33 () {58 vars in 1152 bytes} [Mon Jul 25 05:42:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:42:12.790236 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b74ceb-94b6-4e7c-9b6d-a97a8c7ed366 None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:42:12.799632 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b74ceb-94b6-4e7c-9b6d-a97a8c7ed366 None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:42:12.804603 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-697b1c35-db87-4cfc-b827-2c46c4468eaa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:42:12.805305 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697b1c35-db87-4cfc-b827-2c46c4468eaa admin admin] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:42:12.805590 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697b1c35-db87-4cfc-b827-2c46c4468eaa admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:42:12.805867 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697b1c35-db87-4cfc-b827-2c46c4468eaa admin admin] PATH_INFO: `/v3/projects` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:42:12.806577 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-697b1c35-db87-4cfc-b827-2c46c4468eaa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:42:12.808490 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-697b1c35-db87-4cfc-b827-2c46c4468eaa admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:42:12.834870 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1214/2424] 10.222.0.33 () {60 vars in 1152 bytes} [Mon Jul 25 05:42:12 2022] POST /identity/v3/projects => generated 385 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 05:42:12.839117 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ebe331-7941-4c11-8ea0-8e9c4e4f3e02 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:42:12.849753 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ebe331-7941-4c11-8ea0-8e9c4e4f3e02 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:42:12.908086 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b80595b5-fcee-4a3f-bec9-d0a0c35ccfd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:42:12.908765 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80595b5-fcee-4a3f-bec9-d0a0c35ccfd5 admin admin] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:42:12.909083 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80595b5-fcee-4a3f-bec9-d0a0c35ccfd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:42:12.909364 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80595b5-fcee-4a3f-bec9-d0a0c35ccfd5 admin admin] PATH_INFO: `/v3/users` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:42:12.910058 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b80595b5-fcee-4a3f-bec9-d0a0c35ccfd5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:42:12.911974 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b80595b5-fcee-4a3f-bec9-d0a0c35ccfd5 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:42:12.931717 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1211/2425] 10.222.0.33 () {60 vars in 1143 bytes} [Mon Jul 25 05:42:12 2022] POST /identity/v3/users => generated 293 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 05:42:12.935811 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-697b1c35-db87-4cfc-b827-2c46c4468eaa admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:42:12.946778 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-697b1c35-db87-4cfc-b827-2c46c4468eaa admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:42:12.951395 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e78f4459-4422-4b3a-ae9e-29d9d9dcf155 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:42:12.952053 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78f4459-4422-4b3a-ae9e-29d9d9dcf155 admin admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:42:12.952341 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78f4459-4422-4b3a-ae9e-29d9d9dcf155 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:42:12.952618 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78f4459-4422-4b3a-ae9e-29d9d9dcf155 admin admin] PATH_INFO: `/v3/roles` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:42:12.954009 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e78f4459-4422-4b3a-ae9e-29d9d9dcf155 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:42:12.955847 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e78f4459-4422-4b3a-ae9e-29d9d9dcf155 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:42:12.966055 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1215/2426] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jul 25 05:42:12 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 05:42:12.970169 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b80595b5-fcee-4a3f-bec9-d0a0c35ccfd5 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:42:12.979919 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b80595b5-fcee-4a3f-bec9-d0a0c35ccfd5 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:42:12.984748 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c619a664-26c2-4030-9d50-52ace6eae4b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:42:12.985594 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c619a664-26c2-4030-9d50-52ace6eae4b3 admin admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:42:12.985904 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c619a664-26c2-4030-9d50-52ace6eae4b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:42:12.986187 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c619a664-26c2-4030-9d50-52ace6eae4b3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:42:12.986955 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c619a664-26c2-4030-9d50-52ace6eae4b3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:42:12.988672 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c619a664-26c2-4030-9d50-52ace6eae4b3 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:42:12.998914 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1212/2427] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jul 25 05:42:12 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 05:42:13.003592 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e78f4459-4422-4b3a-ae9e-29d9d9dcf155 admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:42:13.013431 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e78f4459-4422-4b3a-ae9e-29d9d9dcf155 admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:42:13.018107 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e168745d-d5fa-4045-a9b6-adb741ca2821 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:42:13.018969 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e168745d-d5fa-4045-a9b6-adb741ca2821 admin admin] REQUEST_METHOD: `PUT` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:42:13.019254 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e168745d-d5fa-4045-a9b6-adb741ca2821 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:42:13.019529 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e168745d-d5fa-4045-a9b6-adb741ca2821 admin admin] PATH_INFO: `/v3/projects/987ce20ceedd49cb9419cc506c50fce3/users/598f7eff11f545a88d339812207f0f19/roles/5074e74e35f54d4b9bb415c34999a82a` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:42:13.034386 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e168745d-d5fa-4045-a9b6-adb741ca2821 admin admin] RBAC: Authorizing `identity:create_grant(project_id=987ce20ceedd49cb9419cc506c50fce3, user_id=598f7eff11f545a88d339812207f0f19, role_id=5074e74e35f54d4b9bb415c34999a82a)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:42:13.037060 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e168745d-d5fa-4045-a9b6-adb741ca2821 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:42:13.050015 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1216/2428] 10.222.0.33 () {60 vars in 1482 bytes} [Mon Jul 25 05:42:13 2022] PUT /identity/v3/projects/987ce20ceedd49cb9419cc506c50fce3/users/598f7eff11f545a88d339812207f0f19/roles/5074e74e35f54d4b9bb415c34999a82a => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 25 05:42:13.055287 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4954a3d8-4416-48df-b160-3ce0f3e6c797 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:42:13.055670 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4954a3d8-4416-48df-b160-3ce0f3e6c797 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:42:13.056603 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4954a3d8-4416-48df-b160-3ce0f3e6c797 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:42:13.080551 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-4954a3d8-4416-48df-b160-3ce0f3e6c797 None None] MFA Rules not processed for user `1eae540242d94c10b0ad33960a855f08`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:42:13.082045 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-4954a3d8-4416-48df-b160-3ce0f3e6c797 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:42:13.144514 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1213/2429] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 25 05:42:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 05:42:13.154050 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e168745d-d5fa-4045-a9b6-adb741ca2821 admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:42:13.155748 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-e168745d-d5fa-4045-a9b6-adb741ca2821 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:42:13.214343 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e168745d-d5fa-4045-a9b6-adb741ca2821 admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:42:13.218259 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb10e339-72ae-43fd-82fd-054bb65ff6cf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b2931200f99e43d79a50a9f6e02738e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92afba266cce4d1bad4134c351e74be3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:42:13.218907 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb10e339-72ae-43fd-82fd-054bb65ff6cf service neutron] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:42:13.219742 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb10e339-72ae-43fd-82fd-054bb65ff6cf service neutron] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:42:13.220159 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb10e339-72ae-43fd-82fd-054bb65ff6cf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:42:13.237304 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb10e339-72ae-43fd-82fd-054bb65ff6cf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:42:13.238737 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb10e339-72ae-43fd-82fd-054bb65ff6cf service neutron] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:42:13.255532 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1217/2430] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jul 25 05:42:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 05:42:18.174546 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356d00d6-8a57-4a76-8409-97fe16162c72 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:42:18.175099 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356d00d6-8a57-4a76-8409-97fe16162c72 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:42:18.175099 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356d00d6-8a57-4a76-8409-97fe16162c72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:42:18.202736 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-356d00d6-8a57-4a76-8409-97fe16162c72 None None] MFA Rules not processed for user `598f7eff11f545a88d339812207f0f19`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:42:18.204513 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-356d00d6-8a57-4a76-8409-97fe16162c72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:42:18.272194 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1214/2431] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 25 05:42:18 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 05:42:18.276505 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb10e339-72ae-43fd-82fd-054bb65ff6cf service neutron] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:42:18.287480 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb10e339-72ae-43fd-82fd-054bb65ff6cf service neutron] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:42:18.292241 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73809c47-7eea-49be-8c2a-53d2ce32c12f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:42:18.293028 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73809c47-7eea-49be-8c2a-53d2ce32c12f admin admin] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:42:18.293380 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73809c47-7eea-49be-8c2a-53d2ce32c12f admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:42:18.293723 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73809c47-7eea-49be-8c2a-53d2ce32c12f admin admin] PATH_INFO: `/v3/projects` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:42:18.294505 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73809c47-7eea-49be-8c2a-53d2ce32c12f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:42:18.296490 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73809c47-7eea-49be-8c2a-53d2ce32c12f admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:42:18.328304 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1218/2432] 10.222.0.33 () {60 vars in 1152 bytes} [Mon Jul 25 05:42:18 2022] POST /identity/v3/projects => generated 383 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 05:42:18.329885 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-356d00d6-8a57-4a76-8409-97fe16162c72 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:42:18.335523 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:42:18.343033 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-356d00d6-8a57-4a76-8409-97fe16162c72 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:42:18.404194 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d83d5a-d92c-4f92-9c4f-fb050d2989ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:42:18.405051 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d83d5a-d92c-4f92-9c4f-fb050d2989ca admin admin] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:42:18.405440 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d83d5a-d92c-4f92-9c4f-fb050d2989ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:42:18.405814 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d83d5a-d92c-4f92-9c4f-fb050d2989ca admin admin] PATH_INFO: `/v3/users` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:42:18.407237 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47d83d5a-d92c-4f92-9c4f-fb050d2989ca admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:42:18.409352 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47d83d5a-d92c-4f92-9c4f-fb050d2989ca admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:42:18.431474 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1215/2433] 10.222.0.33 () {60 vars in 1143 bytes} [Mon Jul 25 05:42:18 2022] POST /identity/v3/users => generated 298 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 25 05:42:18.435463 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73809c47-7eea-49be-8c2a-53d2ce32c12f admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:42:18.445915 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73809c47-7eea-49be-8c2a-53d2ce32c12f admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:42:18.450546 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aea2f73-0b08-44db-aefb-3bd7e3ca0d4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:42:18.451206 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aea2f73-0b08-44db-aefb-3bd7e3ca0d4f admin admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:42:18.451498 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aea2f73-0b08-44db-aefb-3bd7e3ca0d4f admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:42:18.451779 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aea2f73-0b08-44db-aefb-3bd7e3ca0d4f admin admin] PATH_INFO: `/v3/roles` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:42:18.452510 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aea2f73-0b08-44db-aefb-3bd7e3ca0d4f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:42:18.454797 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aea2f73-0b08-44db-aefb-3bd7e3ca0d4f admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:42:18.465226 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1219/2434] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jul 25 05:42:18 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 25 05:42:18.470024 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d83d5a-d92c-4f92-9c4f-fb050d2989ca admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:42:18.480111 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d83d5a-d92c-4f92-9c4f-fb050d2989ca admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:42:18.485364 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e50f8596-1d29-4409-bf56-587abfeecea7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:42:18.485984 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50f8596-1d29-4409-bf56-587abfeecea7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:42:18.486276 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50f8596-1d29-4409-bf56-587abfeecea7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:42:18.486553 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50f8596-1d29-4409-bf56-587abfeecea7 admin admin] PATH_INFO: `/v3/projects/968e41b62cc34e7198a1e55dd1a7d858/users/6b09e9366f36460fbb934b7b3fe881a5/roles/39e5c666285a42fe8fc2c3afca6260d1` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:42:18.501211 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e50f8596-1d29-4409-bf56-587abfeecea7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=968e41b62cc34e7198a1e55dd1a7d858, user_id=6b09e9366f36460fbb934b7b3fe881a5, role_id=39e5c666285a42fe8fc2c3afca6260d1)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:42:18.503855 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e50f8596-1d29-4409-bf56-587abfeecea7 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:42:18.517059 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1216/2435] 10.222.0.33 () {60 vars in 1482 bytes} [Mon Jul 25 05:42:18 2022] PUT /identity/v3/projects/968e41b62cc34e7198a1e55dd1a7d858/users/6b09e9366f36460fbb934b7b3fe881a5/roles/39e5c666285a42fe8fc2c3afca6260d1 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 05:42:18.521117 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aea2f73-0b08-44db-aefb-3bd7e3ca0d4f admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:42:18.531393 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aea2f73-0b08-44db-aefb-3bd7e3ca0d4f admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:42:18.566837 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:42:18.636876 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e510573-8162-4774-b6d7-973dddce4c1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:42:18.637941 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e510573-8162-4774-b6d7-973dddce4c1d admin admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:42:18.638235 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e510573-8162-4774-b6d7-973dddce4c1d admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:42:18.639246 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e510573-8162-4774-b6d7-973dddce4c1d admin admin] PATH_INFO: `/v3/roles` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:42:18.640002 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e510573-8162-4774-b6d7-973dddce4c1d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:42:18.642234 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e510573-8162-4774-b6d7-973dddce4c1d admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:42:18.657055 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1220/2436] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jul 25 05:42:18 2022] GET /identity/v3/roles => generated 1420 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 05:42:18.659648 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e50f8596-1d29-4409-bf56-587abfeecea7 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:42:18.677765 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e50f8596-1d29-4409-bf56-587abfeecea7 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:42:18.684591 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad1a7700-540a-4658-8a28-c2b69802a752 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:42:18.685693 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1a7700-540a-4658-8a28-c2b69802a752 admin admin] REQUEST_METHOD: `PUT` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:42:18.686081 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1a7700-540a-4658-8a28-c2b69802a752 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:42:18.686456 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1a7700-540a-4658-8a28-c2b69802a752 admin admin] PATH_INFO: `/v3/projects/968e41b62cc34e7198a1e55dd1a7d858/users/6b09e9366f36460fbb934b7b3fe881a5/roles/39e5c666285a42fe8fc2c3afca6260d1` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:42:18.688921 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad1a7700-540a-4658-8a28-c2b69802a752 admin admin] RBAC: Authorizing `identity:create_grant(project_id=968e41b62cc34e7198a1e55dd1a7d858, user_id=6b09e9366f36460fbb934b7b3fe881a5, role_id=39e5c666285a42fe8fc2c3afca6260d1)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:42:18.691673 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad1a7700-540a-4658-8a28-c2b69802a752 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:42:18.711616 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1217/2437] 10.222.0.33 () {60 vars in 1482 bytes} [Mon Jul 25 05:42:18 2022] PUT /identity/v3/projects/968e41b62cc34e7198a1e55dd1a7d858/users/6b09e9366f36460fbb934b7b3fe881a5/roles/39e5c666285a42fe8fc2c3afca6260d1 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 05:42:23.920850 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b10ad1f-7e84-4a5a-b75a-856dbe16f4d0 None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:42:23.921430 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b10ad1f-7e84-4a5a-b75a-856dbe16f4d0 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:42:23.921723 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b10ad1f-7e84-4a5a-b75a-856dbe16f4d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:42:23.958936 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-3b10ad1f-7e84-4a5a-b75a-856dbe16f4d0 None None] MFA Rules not processed for user `6b09e9366f36460fbb934b7b3fe881a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:42:23.960322 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-3b10ad1f-7e84-4a5a-b75a-856dbe16f4d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:42:24.058217 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1221/2438] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 25 05:42:23 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:42:24.075579 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad1a7700-540a-4658-8a28-c2b69802a752 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:42:24.077389 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-ad1a7700-540a-4658-8a28-c2b69802a752 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:42:24.192747 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad1a7700-540a-4658-8a28-c2b69802a752 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:42:24.198423 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a33b555b-91e4-468e-8202-0aa73c908959 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de99e44bdf9c4030adbe9c0efa3831cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92afba266cce4d1bad4134c351e74be3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:42:24.199112 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33b555b-91e4-468e-8202-0aa73c908959 service glance] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:42:24.199405 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33b555b-91e4-468e-8202-0aa73c908959 service glance] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:42:24.199691 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33b555b-91e4-468e-8202-0aa73c908959 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:42:24.365072 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a33b555b-91e4-468e-8202-0aa73c908959 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:42:24.366508 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a33b555b-91e4-468e-8202-0aa73c908959 service glance] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:42:24.414469 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1218/2439] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jul 25 05:42:24 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 339 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 05:42:24.613404 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213b9501-c2ec-44a1-b74c-4b82cdf82999 None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:42:24.613974 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213b9501-c2ec-44a1-b74c-4b82cdf82999 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:42:24.614489 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213b9501-c2ec-44a1-b74c-4b82cdf82999 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:42:24.652956 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-213b9501-c2ec-44a1-b74c-4b82cdf82999 None None] MFA Rules not processed for user `3352531be1b54f76b864383ccc2e5ec2`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:42:24.654324 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-213b9501-c2ec-44a1-b74c-4b82cdf82999 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:42:24.746739 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1222/2440] 10.222.0.33 () {58 vars in 1009 bytes} [Mon Jul 25 05:42:24 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:42:24.838150 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a33b555b-91e4-468e-8202-0aa73c908959 service glance] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:42:24.840150 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-a33b555b-91e4-468e-8202-0aa73c908959 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:42:24.852625 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:42:24.960398 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a33b555b-91e4-468e-8202-0aa73c908959 service glance] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:42:24.971070 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f99086-f7c5-44ec-bcde-4613f83ca5fb service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb135a8d5d7444cfbfa336115ba2ba44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92afba266cce4d1bad4134c351e74be3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:42:24.972604 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f99086-f7c5-44ec-bcde-4613f83ca5fb service swift] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:42:24.973669 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f99086-f7c5-44ec-bcde-4613f83ca5fb service swift] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:42:24.974189 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f99086-f7c5-44ec-bcde-4613f83ca5fb service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:42:24.996615 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5f99086-f7c5-44ec-bcde-4613f83ca5fb service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:42:24.999336 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5f99086-f7c5-44ec-bcde-4613f83ca5fb service swift] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:42:25.033087 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1219/2441] 10.222.0.33 () {60 vars in 1384 bytes} [Mon Jul 25 05:42:24 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 195 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 25 05:42:29.969903 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106e7176-ce3c-4927-b140-5488c2419d92 None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:42:29.971053 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106e7176-ce3c-4927-b140-5488c2419d92 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:42:29.971519 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106e7176-ce3c-4927-b140-5488c2419d92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:42:29.993209 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-106e7176-ce3c-4927-b140-5488c2419d92 None None] MFA Rules not processed for user `69f3276517de4f889dddd3180436ede9`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:42:29.995143 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-106e7176-ce3c-4927-b140-5488c2419d92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:42:30.045926 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:42:30.059922 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1223/2442] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 25 05:42:29 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:42:30.198817 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f99086-f7c5-44ec-bcde-4613f83ca5fb service swift] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:42:30.199857 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-d5f99086-f7c5-44ec-bcde-4613f83ca5fb service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:42:30.265588 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f99086-f7c5-44ec-bcde-4613f83ca5fb service swift] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:42:30.270113 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b254c7-2db6-4b24-8551-2a34f472ffe3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b2931200f99e43d79a50a9f6e02738e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92afba266cce4d1bad4134c351e74be3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:42:30.270775 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b254c7-2db6-4b24-8551-2a34f472ffe3 service neutron] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:42:30.271124 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b254c7-2db6-4b24-8551-2a34f472ffe3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:42:30.271456 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b254c7-2db6-4b24-8551-2a34f472ffe3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:42:30.273834 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-45b254c7-2db6-4b24-8551-2a34f472ffe3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:42:30.287318 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45b254c7-2db6-4b24-8551-2a34f472ffe3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:42:30.288769 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45b254c7-2db6-4b24-8551-2a34f472ffe3 service neutron] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:42:30.299035 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1220/2443] 10.222.0.33 () {60 vars in 1396 bytes} [Mon Jul 25 05:42:30 2022] GET /identity/v3/auth/tokens?allow_expired=1 => generated 3252 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 05:42:30.512445 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-106e7176-ce3c-4927-b140-5488c2419d92 None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:42:30.530617 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-106e7176-ce3c-4927-b140-5488c2419d92 None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:42:30.539720 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c3c0224-0758-41ae-909c-c964fdd4ea86 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b2931200f99e43d79a50a9f6e02738e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92afba266cce4d1bad4134c351e74be3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:42:30.540394 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3c0224-0758-41ae-909c-c964fdd4ea86 service neutron] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:42:30.540679 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3c0224-0758-41ae-909c-c964fdd4ea86 service neutron] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:42:30.541010 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3c0224-0758-41ae-909c-c964fdd4ea86 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:42:30.543410 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-3c3c0224-0758-41ae-909c-c964fdd4ea86 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:42:30.564928 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c3c0224-0758-41ae-909c-c964fdd4ea86 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:42:30.566394 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c3c0224-0758-41ae-909c-c964fdd4ea86 service neutron] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:42:30.575667 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1224/2444] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jul 25 05:42:30 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 64 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 05:42:30.973864 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b254c7-2db6-4b24-8551-2a34f472ffe3 service neutron] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:42:30.975541 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-45b254c7-2db6-4b24-8551-2a34f472ffe3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:42:31.060864 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b254c7-2db6-4b24-8551-2a34f472ffe3 service neutron] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:42:31.066803 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d918d850-9496-4b63-81c1-8b6dc5836b71 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '317790016e7047fe96af73925de5d872', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92afba266cce4d1bad4134c351e74be3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:42:31.067564 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d918d850-9496-4b63-81c1-8b6dc5836b71 service placement] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:42:31.067915 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d918d850-9496-4b63-81c1-8b6dc5836b71 service placement] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:42:31.068258 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d918d850-9496-4b63-81c1-8b6dc5836b71 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:42:31.070214 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-d918d850-9496-4b63-81c1-8b6dc5836b71 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:42:31.084684 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d918d850-9496-4b63-81c1-8b6dc5836b71 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:42:31.086440 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d918d850-9496-4b63-81c1-8b6dc5836b71 service placement] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:42:31.098322 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1221/2445] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jul 25 05:42:30 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 05:42:32.743906 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c3c0224-0758-41ae-909c-c964fdd4ea86 service neutron] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:42:32.745595 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-3c3c0224-0758-41ae-909c-c964fdd4ea86 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:42:32.862118 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c3c0224-0758-41ae-909c-c964fdd4ea86 service neutron] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:42:32.869042 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd38095-6f9b-443b-9f6e-859058a37d43 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cd482a61e8a420db45bb532e5dae6e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92afba266cce4d1bad4134c351e74be3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:42:32.869916 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd38095-6f9b-443b-9f6e-859058a37d43 service nova] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:42:32.870361 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd38095-6f9b-443b-9f6e-859058a37d43 service nova] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:42:32.870794 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd38095-6f9b-443b-9f6e-859058a37d43 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:42:32.872892 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-cdd38095-6f9b-443b-9f6e-859058a37d43 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:42:32.890164 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdd38095-6f9b-443b-9f6e-859058a37d43 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:42:32.892833 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdd38095-6f9b-443b-9f6e-859058a37d43 service nova] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:42:32.907100 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1225/2446] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jul 25 05:42:32 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 164 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 25 05:42:33.898877 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d918d850-9496-4b63-81c1-8b6dc5836b71 service placement] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:42:33.913970 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d918d850-9496-4b63-81c1-8b6dc5836b71 service placement] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:42:33.917859 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2593666e-b204-41d2-b499-d8f25d352b73 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b2931200f99e43d79a50a9f6e02738e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92afba266cce4d1bad4134c351e74be3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:42:33.918763 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2593666e-b204-41d2-b499-d8f25d352b73 service neutron] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:42:33.919153 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2593666e-b204-41d2-b499-d8f25d352b73 service neutron] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:42:33.919522 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2593666e-b204-41d2-b499-d8f25d352b73 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:42:33.921525 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-2593666e-b204-41d2-b499-d8f25d352b73 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:42:33.932572 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2593666e-b204-41d2-b499-d8f25d352b73 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:42:33.934071 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2593666e-b204-41d2-b499-d8f25d352b73 service neutron] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:42:33.947302 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1222/2447] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jul 25 05:42:33 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 05:42:36.543518 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d7682f-96e4-4f09-909d-362a2a3b787a None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:42:36.544142 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d7682f-96e4-4f09-909d-362a2a3b787a None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:42:36.544142 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d7682f-96e4-4f09-909d-362a2a3b787a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:42:36.563520 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-17d7682f-96e4-4f09-909d-362a2a3b787a None None] MFA Rules not processed for user `598f7eff11f545a88d339812207f0f19`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:42:36.564835 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-17d7682f-96e4-4f09-909d-362a2a3b787a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:42:36.571880 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1226/2448] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 25 05:42:36 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:42:43.088752 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d052f3-71bc-4642-ad33-870867cb22b2 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:42:43.089327 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d052f3-71bc-4642-ad33-870867cb22b2 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:42:43.089327 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d052f3-71bc-4642-ad33-870867cb22b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:42:43.117039 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-a5d052f3-71bc-4642-ad33-870867cb22b2 None None] MFA Rules not processed for user `3352531be1b54f76b864383ccc2e5ec2`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:42:43.118373 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-a5d052f3-71bc-4642-ad33-870867cb22b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:42:43.126022 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1223/2449] 10.222.0.33 () {58 vars in 1009 bytes} [Mon Jul 25 05:42:43 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:42:43.140049 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d7682f-96e4-4f09-909d-362a2a3b787a None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:42:43.154180 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d7682f-96e4-4f09-909d-362a2a3b787a None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:42:43.159712 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af0c6a6c-b341-462b-8c9c-91860e108766 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb135a8d5d7444cfbfa336115ba2ba44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92afba266cce4d1bad4134c351e74be3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:42:43.160566 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0c6a6c-b341-462b-8c9c-91860e108766 service swift] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:42:43.160992 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0c6a6c-b341-462b-8c9c-91860e108766 service swift] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:42:43.161368 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0c6a6c-b341-462b-8c9c-91860e108766 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:42:43.170756 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af0c6a6c-b341-462b-8c9c-91860e108766 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:42:43.173442 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af0c6a6c-b341-462b-8c9c-91860e108766 service swift] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:42:43.185167 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1227/2450] 10.222.0.33 () {60 vars in 1384 bytes} [Mon Jul 25 05:42:43 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jul 25 05:42:43.721643 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d052f3-71bc-4642-ad33-870867cb22b2 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:42:43.736729 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d052f3-71bc-4642-ad33-870867cb22b2 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:42:43.742886 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-572c2eb1-2a7e-4e7d-beb9-0644d5796545 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cd482a61e8a420db45bb532e5dae6e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92afba266cce4d1bad4134c351e74be3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:42:43.743967 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572c2eb1-2a7e-4e7d-beb9-0644d5796545 service nova] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:42:43.744224 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572c2eb1-2a7e-4e7d-beb9-0644d5796545 service nova] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:42:43.744625 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572c2eb1-2a7e-4e7d-beb9-0644d5796545 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:42:43.746743 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-572c2eb1-2a7e-4e7d-beb9-0644d5796545 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:42:43.760334 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-572c2eb1-2a7e-4e7d-beb9-0644d5796545 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:42:43.762009 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-572c2eb1-2a7e-4e7d-beb9-0644d5796545 service nova] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:42:43.768353 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:42:43.793938 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1224/2451] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jul 25 05:42:43 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 73 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 05:42:45.991188 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af0c6a6c-b341-462b-8c9c-91860e108766 service swift] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:42:46.007136 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af0c6a6c-b341-462b-8c9c-91860e108766 service swift] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:42:46.020552 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314be1c5-beae-4fbb-9812-7414f52379e7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b2931200f99e43d79a50a9f6e02738e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92afba266cce4d1bad4134c351e74be3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:42:46.021758 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314be1c5-beae-4fbb-9812-7414f52379e7 service neutron] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:42:46.022284 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314be1c5-beae-4fbb-9812-7414f52379e7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:42:46.022948 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314be1c5-beae-4fbb-9812-7414f52379e7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:42:46.034575 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-314be1c5-beae-4fbb-9812-7414f52379e7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:42:46.223460 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-314be1c5-beae-4fbb-9812-7414f52379e7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:42:46.226028 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-314be1c5-beae-4fbb-9812-7414f52379e7 service neutron] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:42:46.271570 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1228/2452] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jul 25 05:42:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 281 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 05:42:47.724561 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3204fed4-3c0b-40e1-a5e6-4482c1cf580e None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:42:47.725132 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3204fed4-3c0b-40e1-a5e6-4482c1cf580e None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:42:47.725309 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3204fed4-3c0b-40e1-a5e6-4482c1cf580e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:42:47.782706 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-3204fed4-3c0b-40e1-a5e6-4482c1cf580e None None] MFA Rules not processed for user `1f815f111bf74739969736acea131927`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:42:47.792568 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-3204fed4-3c0b-40e1-a5e6-4482c1cf580e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:42:47.921353 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1225/2453] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 25 05:42:47 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:42:50.039594 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314be1c5-beae-4fbb-9812-7414f52379e7 service neutron] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:42:50.061332 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314be1c5-beae-4fbb-9812-7414f52379e7 service neutron] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:42:50.078733 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b736c3d-786d-424a-90be-0a2f42915c33 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cd482a61e8a420db45bb532e5dae6e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92afba266cce4d1bad4134c351e74be3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:42:50.080075 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b736c3d-786d-424a-90be-0a2f42915c33 service nova] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:42:50.080444 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b736c3d-786d-424a-90be-0a2f42915c33 service nova] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:42:50.080761 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b736c3d-786d-424a-90be-0a2f42915c33 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:42:50.082667 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-3b736c3d-786d-424a-90be-0a2f42915c33 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:42:50.120972 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b736c3d-786d-424a-90be-0a2f42915c33 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:42:50.122763 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b736c3d-786d-424a-90be-0a2f42915c33 service nova] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:42:50.179620 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1229/2454] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jul 25 05:42:50 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 05:42:51.592836 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758850dc-39f5-46a7-a8f4-f9a018e0b05b None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:42:51.593679 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758850dc-39f5-46a7-a8f4-f9a018e0b05b None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:42:51.594073 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758850dc-39f5-46a7-a8f4-f9a018e0b05b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:42:51.632612 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-758850dc-39f5-46a7-a8f4-f9a018e0b05b None None] MFA Rules not processed for user `c1ae82a1862048cb9038b7c76190da8a`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:42:51.636911 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-758850dc-39f5-46a7-a8f4-f9a018e0b05b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:42:51.840539 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1226/2455] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 25 05:42:51 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 249 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:42:54.639024 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b736c3d-786d-424a-90be-0a2f42915c33 service nova] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:42:54.640468 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-3b736c3d-786d-424a-90be-0a2f42915c33 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:42:54.669374 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b736c3d-786d-424a-90be-0a2f42915c33 service nova] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:42:54.675890 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-836d85b1-7a00-41b3-b0ef-772d0ba382a4 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '317790016e7047fe96af73925de5d872', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92afba266cce4d1bad4134c351e74be3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:42:54.676850 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836d85b1-7a00-41b3-b0ef-772d0ba382a4 service placement] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:42:54.677440 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836d85b1-7a00-41b3-b0ef-772d0ba382a4 service placement] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:42:54.677876 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836d85b1-7a00-41b3-b0ef-772d0ba382a4 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:42:54.679987 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-836d85b1-7a00-41b3-b0ef-772d0ba382a4 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:42:54.699609 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-836d85b1-7a00-41b3-b0ef-772d0ba382a4 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:42:54.701547 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-836d85b1-7a00-41b3-b0ef-772d0ba382a4 service placement] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:42:54.718079 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1230/2456] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jul 25 05:42:54 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 79 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 05:42:54.843804 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-758850dc-39f5-46a7-a8f4-f9a018e0b05b None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:42:54.845928 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-758850dc-39f5-46a7-a8f4-f9a018e0b05b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:42:54.862013 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-758850dc-39f5-46a7-a8f4-f9a018e0b05b None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:42:54.866673 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-551572cf-75b8-431a-9d6f-c26748301570 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:42:54.867612 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551572cf-75b8-431a-9d6f-c26748301570 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:42:54.868035 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551572cf-75b8-431a-9d6f-c26748301570 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:42:54.868437 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551572cf-75b8-431a-9d6f-c26748301570 admin admin] PATH_INFO: `/v3/users/37c1bb8500f74013b073febfc0d96217` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:42:54.869695 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-551572cf-75b8-431a-9d6f-c26748301570 admin admin] RBAC: Authorizing `identity:delete_user(user_id=37c1bb8500f74013b073febfc0d96217)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:42:54.871658 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-551572cf-75b8-431a-9d6f-c26748301570 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:42:54.919030 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-551572cf-75b8-431a-9d6f-c26748301570 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '37c1bb8500f74013b073febfc0d96217'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:42:54.919699 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-551572cf-75b8-431a-9d6f-c26748301570 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '37c1bb8500f74013b073febfc0d96217'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:42:54.926547 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-551572cf-75b8-431a-9d6f-c26748301570 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '37c1bb8500f74013b073febfc0d96217'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:42:54.940315 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-551572cf-75b8-431a-9d6f-c26748301570 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '37c1bb8500f74013b073febfc0d96217'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:42:54.941022 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-551572cf-75b8-431a-9d6f-c26748301570 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '37c1bb8500f74013b073febfc0d96217'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:42:54.953262 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1227/2457] 10.222.0.33 () {58 vars in 1223 bytes} [Mon Jul 25 05:42:54 2022] DELETE /identity/v3/users/37c1bb8500f74013b073febfc0d96217 => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 05:42:55.145737 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-836d85b1-7a00-41b3-b0ef-772d0ba382a4 service placement] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:42:55.156672 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-836d85b1-7a00-41b3-b0ef-772d0ba382a4 service placement] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:42:55.169413 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:42:55.215784 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c95156-16ab-43cb-bd8b-97cb8d514d13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:42:55.216649 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c95156-16ab-43cb-bd8b-97cb8d514d13 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:42:55.216964 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c95156-16ab-43cb-bd8b-97cb8d514d13 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:42:55.217243 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c95156-16ab-43cb-bd8b-97cb8d514d13 admin admin] PATH_INFO: `/v3/projects/c782b01e8e484fe8bb710aeddc0192d1` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:42:55.218711 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49c95156-16ab-43cb-bd8b-97cb8d514d13 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c782b01e8e484fe8bb710aeddc0192d1)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:42:55.220718 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49c95156-16ab-43cb-bd8b-97cb8d514d13 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:42:55.268095 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-49c95156-16ab-43cb-bd8b-97cb8d514d13 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c782b01e8e484fe8bb710aeddc0192d1', 'request_id': 'req-49c95156-16ab-43cb-bd8b-97cb8d514d13'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:42:55.273522 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-49c95156-16ab-43cb-bd8b-97cb8d514d13 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c782b01e8e484fe8bb710aeddc0192d1', 'request_id': 'req-49c95156-16ab-43cb-bd8b-97cb8d514d13'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:42:55.279613 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-49c95156-16ab-43cb-bd8b-97cb8d514d13 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c782b01e8e484fe8bb710aeddc0192d1', 'request_id': 'req-49c95156-16ab-43cb-bd8b-97cb8d514d13'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:42:55.291794 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-49c95156-16ab-43cb-bd8b-97cb8d514d13 admin admin] The token cache is being invalidate because project c782b01e8e484fe8bb710aeddc0192d1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80248) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 05:42:55.292086 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-49c95156-16ab-43cb-bd8b-97cb8d514d13 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:42:55.292709 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-49c95156-16ab-43cb-bd8b-97cb8d514d13 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:42:55.294297 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1231/2458] 10.222.0.33 () {58 vars in 1232 bytes} [Mon Jul 25 05:42:55 2022] DELETE /identity/v3/projects/c782b01e8e484fe8bb710aeddc0192d1 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 05:42:55.300886 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe67a4e8-17bb-4ab2-aff3-22ba2814caa7 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:42:55.301187 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe67a4e8-17bb-4ab2-aff3-22ba2814caa7 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:42:55.301489 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe67a4e8-17bb-4ab2-aff3-22ba2814caa7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:42:55.319783 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-fe67a4e8-17bb-4ab2-aff3-22ba2814caa7 None None] MFA Rules not processed for user `1eae540242d94c10b0ad33960a855f08`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:42:55.321042 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-fe67a4e8-17bb-4ab2-aff3-22ba2814caa7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:42:55.334759 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:42:55.384200 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1228/2459] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 25 05:42:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:42:55.392044 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456eb14b-6e7d-40a4-8cd4-541311daa3d2 None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:42:55.392367 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456eb14b-6e7d-40a4-8cd4-541311daa3d2 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:42:55.392652 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456eb14b-6e7d-40a4-8cd4-541311daa3d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:42:55.410675 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-456eb14b-6e7d-40a4-8cd4-541311daa3d2 None None] MFA Rules not processed for user `1eae540242d94c10b0ad33960a855f08`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:42:55.411878 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-456eb14b-6e7d-40a4-8cd4-541311daa3d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:42:55.419757 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1232/2460] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 25 05:42:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:42:55.424582 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe67a4e8-17bb-4ab2-aff3-22ba2814caa7 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:42:55.434478 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe67a4e8-17bb-4ab2-aff3-22ba2814caa7 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:42:55.440232 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f3f3d4-088d-4db3-9f35-1f7b7eee6557 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:42:55.440922 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f3f3d4-088d-4db3-9f35-1f7b7eee6557 admin admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:42:55.441215 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f3f3d4-088d-4db3-9f35-1f7b7eee6557 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:42:55.441499 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f3f3d4-088d-4db3-9f35-1f7b7eee6557 admin admin] PATH_INFO: `/v3/domains` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:42:55.442244 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02f3f3d4-088d-4db3-9f35-1f7b7eee6557 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:42:55.442558 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02f3f3d4-088d-4db3-9f35-1f7b7eee6557 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:42:55.444153 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02f3f3d4-088d-4db3-9f35-1f7b7eee6557 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:42:55.456073 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1229/2461] 10.222.0.33 () {58 vars in 1152 bytes} [Mon Jul 25 05:42:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:42:55.460322 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-456eb14b-6e7d-40a4-8cd4-541311daa3d2 None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:42:55.469941 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-456eb14b-6e7d-40a4-8cd4-541311daa3d2 None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:42:55.475023 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-237ed299-267d-4cd0-900d-e0a2fcec0d76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:42:55.475680 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237ed299-267d-4cd0-900d-e0a2fcec0d76 admin admin] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:42:55.475963 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237ed299-267d-4cd0-900d-e0a2fcec0d76 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:42:55.476240 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237ed299-267d-4cd0-900d-e0a2fcec0d76 admin admin] PATH_INFO: `/v3/projects` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:42:55.476961 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-237ed299-267d-4cd0-900d-e0a2fcec0d76 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:42:55.478848 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-237ed299-267d-4cd0-900d-e0a2fcec0d76 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:42:55.504598 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1233/2462] 10.222.0.33 () {60 vars in 1152 bytes} [Mon Jul 25 05:42:55 2022] POST /identity/v3/projects => generated 389 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 05:42:55.508082 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f3f3d4-088d-4db3-9f35-1f7b7eee6557 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:42:55.518075 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f3f3d4-088d-4db3-9f35-1f7b7eee6557 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:42:55.576604 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dee1496-0137-4fb8-ab90-c016d1587c3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:42:55.577355 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dee1496-0137-4fb8-ab90-c016d1587c3f admin admin] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:42:55.577640 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dee1496-0137-4fb8-ab90-c016d1587c3f admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:42:55.577918 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dee1496-0137-4fb8-ab90-c016d1587c3f admin admin] PATH_INFO: `/v3/users` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:42:55.578637 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dee1496-0137-4fb8-ab90-c016d1587c3f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:42:55.580545 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dee1496-0137-4fb8-ab90-c016d1587c3f admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:42:55.606701 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1230/2463] 10.222.0.33 () {60 vars in 1143 bytes} [Mon Jul 25 05:42:55 2022] POST /identity/v3/users => generated 295 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 05:42:55.607638 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-237ed299-267d-4cd0-900d-e0a2fcec0d76 admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:42:55.625881 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-237ed299-267d-4cd0-900d-e0a2fcec0d76 admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:42:55.634530 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d52aa50e-dc78-482a-8a9e-6292da19bfa8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:42:55.635346 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52aa50e-dc78-482a-8a9e-6292da19bfa8 admin admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:42:55.635626 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52aa50e-dc78-482a-8a9e-6292da19bfa8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:42:55.635903 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52aa50e-dc78-482a-8a9e-6292da19bfa8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:42:55.636644 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d52aa50e-dc78-482a-8a9e-6292da19bfa8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:42:55.638518 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d52aa50e-dc78-482a-8a9e-6292da19bfa8 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:42:55.654963 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1234/2464] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jul 25 05:42:55 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 05:42:55.659257 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dee1496-0137-4fb8-ab90-c016d1587c3f admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:42:55.670733 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dee1496-0137-4fb8-ab90-c016d1587c3f admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:42:55.677543 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-263b8650-c644-47cd-8199-36627f4343a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:42:55.678498 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263b8650-c644-47cd-8199-36627f4343a5 admin admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:42:55.678793 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263b8650-c644-47cd-8199-36627f4343a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:42:55.679135 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263b8650-c644-47cd-8199-36627f4343a5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:42:55.680667 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-263b8650-c644-47cd-8199-36627f4343a5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:42:55.682644 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-263b8650-c644-47cd-8199-36627f4343a5 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:42:55.701882 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d52aa50e-dc78-482a-8a9e-6292da19bfa8 admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:42:55.708825 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1231/2465] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jul 25 05:42:55 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 05:42:55.715094 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d52aa50e-dc78-482a-8a9e-6292da19bfa8 admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:42:55.721371 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b39b37f-6248-4847-a4d0-1ca85da5f6bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:42:55.722572 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b39b37f-6248-4847-a4d0-1ca85da5f6bd admin admin] REQUEST_METHOD: `PUT` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:42:55.723055 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b39b37f-6248-4847-a4d0-1ca85da5f6bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:42:55.723762 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b39b37f-6248-4847-a4d0-1ca85da5f6bd admin admin] PATH_INFO: `/v3/projects/07879d074bee4b36b1f2f661dc4aeb7f/users/6973623dc52c402ab51dbf1a7169d3e9/roles/5074e74e35f54d4b9bb415c34999a82a` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:42:55.744182 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b39b37f-6248-4847-a4d0-1ca85da5f6bd admin admin] RBAC: Authorizing `identity:create_grant(project_id=07879d074bee4b36b1f2f661dc4aeb7f, user_id=6973623dc52c402ab51dbf1a7169d3e9, role_id=5074e74e35f54d4b9bb415c34999a82a)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:42:55.748160 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b39b37f-6248-4847-a4d0-1ca85da5f6bd admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:42:55.771930 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1235/2466] 10.222.0.33 () {60 vars in 1482 bytes} [Mon Jul 25 05:42:55 2022] PUT /identity/v3/projects/07879d074bee4b36b1f2f661dc4aeb7f/users/6973623dc52c402ab51dbf1a7169d3e9/roles/5074e74e35f54d4b9bb415c34999a82a => generated 0 bytes in 70 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 05:42:55.775203 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c98cbbc-461b-4ace-b905-a0de8285d47f None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:42:55.775595 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c98cbbc-461b-4ace-b905-a0de8285d47f None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:42:55.775971 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c98cbbc-461b-4ace-b905-a0de8285d47f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:42:55.804952 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-5c98cbbc-461b-4ace-b905-a0de8285d47f None None] MFA Rules not processed for user `1eae540242d94c10b0ad33960a855f08`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:42:55.806879 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-5c98cbbc-461b-4ace-b905-a0de8285d47f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:42:55.876327 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1232/2467] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 25 05:42:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:42:55.886259 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b39b37f-6248-4847-a4d0-1ca85da5f6bd admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:42:55.887992 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-9b39b37f-6248-4847-a4d0-1ca85da5f6bd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:42:55.933639 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:42:55.946999 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b39b37f-6248-4847-a4d0-1ca85da5f6bd admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:42:55.950657 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f438c649-2d25-47ed-90a8-3554b721a53e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b2931200f99e43d79a50a9f6e02738e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92afba266cce4d1bad4134c351e74be3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:42:55.951296 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f438c649-2d25-47ed-90a8-3554b721a53e service neutron] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:42:55.951579 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f438c649-2d25-47ed-90a8-3554b721a53e service neutron] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:42:55.951854 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f438c649-2d25-47ed-90a8-3554b721a53e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:42:55.961756 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f438c649-2d25-47ed-90a8-3554b721a53e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:42:55.963178 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f438c649-2d25-47ed-90a8-3554b721a53e service neutron] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:42:55.973783 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1236/2468] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jul 25 05:42:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 05:43:03.550070 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241107b0-4f80-4566-9c34-7cee2e4e98a2 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:43:03.550626 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241107b0-4f80-4566-9c34-7cee2e4e98a2 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:43:03.550626 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241107b0-4f80-4566-9c34-7cee2e4e98a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:43:03.581085 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-241107b0-4f80-4566-9c34-7cee2e4e98a2 None None] MFA Rules not processed for user `6973623dc52c402ab51dbf1a7169d3e9`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:43:03.582612 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-241107b0-4f80-4566-9c34-7cee2e4e98a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:43:03.599608 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:43:03.665562 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1233/2469] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 25 05:43:03 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:43:03.683895 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f438c649-2d25-47ed-90a8-3554b721a53e service neutron] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:43:03.685367 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-f438c649-2d25-47ed-90a8-3554b721a53e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:43:03.761043 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f438c649-2d25-47ed-90a8-3554b721a53e service neutron] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:43:03.766517 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-818f962a-93df-4e68-9054-90227c58a1dc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cd482a61e8a420db45bb532e5dae6e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92afba266cce4d1bad4134c351e74be3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:43:03.767167 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818f962a-93df-4e68-9054-90227c58a1dc service nova] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:43:03.767650 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818f962a-93df-4e68-9054-90227c58a1dc service nova] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:43:03.768405 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818f962a-93df-4e68-9054-90227c58a1dc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:43:03.776459 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-818f962a-93df-4e68-9054-90227c58a1dc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:43:03.778120 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-818f962a-93df-4e68-9054-90227c58a1dc service nova] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:43:03.788021 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1237/2470] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jul 25 05:43:03 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 05:43:05.485403 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f1652e-fe71-4e8e-b3e6-bb36efec7758 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:43:05.485951 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f1652e-fe71-4e8e-b3e6-bb36efec7758 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:43:05.486053 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f1652e-fe71-4e8e-b3e6-bb36efec7758 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:43:05.515318 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-10f1652e-fe71-4e8e-b3e6-bb36efec7758 None None] MFA Rules not processed for user `6973623dc52c402ab51dbf1a7169d3e9`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:43:05.516975 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-10f1652e-fe71-4e8e-b3e6-bb36efec7758 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:43:05.542992 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1234/2471] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 25 05:43:05 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 59 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:43:06.004857 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-818f962a-93df-4e68-9054-90227c58a1dc service nova] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:43:06.006428 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-818f962a-93df-4e68-9054-90227c58a1dc service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:43:06.021514 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-818f962a-93df-4e68-9054-90227c58a1dc service nova] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:43:06.026829 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a56100e4-af77-40cd-b27c-11b998ebb480 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b2931200f99e43d79a50a9f6e02738e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92afba266cce4d1bad4134c351e74be3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:43:06.027956 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56100e4-af77-40cd-b27c-11b998ebb480 service neutron] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:43:06.028444 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56100e4-af77-40cd-b27c-11b998ebb480 service neutron] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:43:06.029860 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56100e4-af77-40cd-b27c-11b998ebb480 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:43:06.032536 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-a56100e4-af77-40cd-b27c-11b998ebb480 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:43:06.053269 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a56100e4-af77-40cd-b27c-11b998ebb480 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:43:06.055806 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a56100e4-af77-40cd-b27c-11b998ebb480 service neutron] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:43:06.071830 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1238/2472] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jul 25 05:43:06 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 68 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 05:43:15.210749 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f1652e-fe71-4e8e-b3e6-bb36efec7758 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:43:15.236489 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f1652e-fe71-4e8e-b3e6-bb36efec7758 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:43:15.240834 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc05fc6-0812-4fb2-9390-75c44001008f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b2931200f99e43d79a50a9f6e02738e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92afba266cce4d1bad4134c351e74be3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:43:15.241676 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc05fc6-0812-4fb2-9390-75c44001008f service neutron] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:43:15.242157 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc05fc6-0812-4fb2-9390-75c44001008f service neutron] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:43:15.242605 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc05fc6-0812-4fb2-9390-75c44001008f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:43:15.244633 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-8dc05fc6-0812-4fb2-9390-75c44001008f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:43:15.344933 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dc05fc6-0812-4fb2-9390-75c44001008f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:43:15.346951 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dc05fc6-0812-4fb2-9390-75c44001008f service neutron] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:43:15.358954 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1235/2473] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jul 25 05:43:15 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 149 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 05:43:16.067287 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=47 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:43:25.352448 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=53 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:43:56.306878 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a56100e4-af77-40cd-b27c-11b998ebb480 service neutron] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:43:56.315352 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a56100e4-af77-40cd-b27c-11b998ebb480 service neutron] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:43:56.320954 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc7b585d-bcc8-4b30-9b57-d0cec77f7ddb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b2931200f99e43d79a50a9f6e02738e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92afba266cce4d1bad4134c351e74be3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:43:56.321685 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7b585d-bcc8-4b30-9b57-d0cec77f7ddb service neutron] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:43:56.322037 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7b585d-bcc8-4b30-9b57-d0cec77f7ddb service neutron] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:43:56.322383 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7b585d-bcc8-4b30-9b57-d0cec77f7ddb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:43:56.324228 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-bc7b585d-bcc8-4b30-9b57-d0cec77f7ddb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:43:56.337712 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc7b585d-bcc8-4b30-9b57-d0cec77f7ddb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:43:56.339177 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc7b585d-bcc8-4b30-9b57-d0cec77f7ddb service neutron] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:43:56.350146 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1239/2474] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jul 25 05:43:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 05:43:57.506792 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc05fc6-0812-4fb2-9390-75c44001008f service neutron] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:43:57.523485 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc05fc6-0812-4fb2-9390-75c44001008f service neutron] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:43:57.529811 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aab728e-a7e9-48f1-9daf-0fce6ca5195b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cd482a61e8a420db45bb532e5dae6e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92afba266cce4d1bad4134c351e74be3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:43:57.530808 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aab728e-a7e9-48f1-9daf-0fce6ca5195b service nova] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:43:57.531488 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aab728e-a7e9-48f1-9daf-0fce6ca5195b service nova] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:43:57.531990 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aab728e-a7e9-48f1-9daf-0fce6ca5195b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:43:57.536073 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-6aab728e-a7e9-48f1-9daf-0fce6ca5195b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:43:57.614904 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aab728e-a7e9-48f1-9daf-0fce6ca5195b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:43:57.617428 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aab728e-a7e9-48f1-9daf-0fce6ca5195b service nova] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:43:57.629214 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1236/2475] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jul 25 05:43:57 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 05:43:59.666038 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc7b585d-bcc8-4b30-9b57-d0cec77f7ddb service neutron] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:43:59.667526 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-bc7b585d-bcc8-4b30-9b57-d0cec77f7ddb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:43:59.684603 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc7b585d-bcc8-4b30-9b57-d0cec77f7ddb service neutron] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:43:59.689401 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db8cee6-b594-464a-bc12-a70304ce0dac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:43:59.690291 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db8cee6-b594-464a-bc12-a70304ce0dac admin admin] REQUEST_METHOD: `DELETE` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:43:59.690611 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db8cee6-b594-464a-bc12-a70304ce0dac admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:43:59.690891 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db8cee6-b594-464a-bc12-a70304ce0dac admin admin] PATH_INFO: `/v3/users/69f3276517de4f889dddd3180436ede9` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:43:59.692334 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5db8cee6-b594-464a-bc12-a70304ce0dac admin admin] RBAC: Authorizing `identity:delete_user(user_id=69f3276517de4f889dddd3180436ede9)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:43:59.694355 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5db8cee6-b594-464a-bc12-a70304ce0dac admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:43:59.742163 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-5db8cee6-b594-464a-bc12-a70304ce0dac admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '69f3276517de4f889dddd3180436ede9'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:43:59.754392 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-5db8cee6-b594-464a-bc12-a70304ce0dac admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '69f3276517de4f889dddd3180436ede9'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:43:59.755125 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-5db8cee6-b594-464a-bc12-a70304ce0dac admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '69f3276517de4f889dddd3180436ede9'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:43:59.762591 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-5db8cee6-b594-464a-bc12-a70304ce0dac admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '69f3276517de4f889dddd3180436ede9'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:43:59.763274 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-5db8cee6-b594-464a-bc12-a70304ce0dac admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '69f3276517de4f889dddd3180436ede9'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:43:59.776349 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1240/2476] 10.222.0.33 () {58 vars in 1223 bytes} [Mon Jul 25 05:43:59 2022] DELETE /identity/v3/users/69f3276517de4f889dddd3180436ede9 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 05:43:59.950300 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aab728e-a7e9-48f1-9daf-0fce6ca5195b service nova] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:43:59.966396 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aab728e-a7e9-48f1-9daf-0fce6ca5195b service nova] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:44:00.043396 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54569807-7634-4b25-8292-b80cf081a0bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:44:00.044259 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54569807-7634-4b25-8292-b80cf081a0bb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:44:00.044546 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54569807-7634-4b25-8292-b80cf081a0bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:44:00.044855 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54569807-7634-4b25-8292-b80cf081a0bb admin admin] PATH_INFO: `/v3/projects/42066c73ac8f4e5c883204dfc66245c0` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:44:00.046235 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54569807-7634-4b25-8292-b80cf081a0bb admin admin] RBAC: Authorizing `identity:delete_project(project_id=42066c73ac8f4e5c883204dfc66245c0)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:44:00.048169 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54569807-7634-4b25-8292-b80cf081a0bb admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:44:00.098766 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-54569807-7634-4b25-8292-b80cf081a0bb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '42066c73ac8f4e5c883204dfc66245c0', 'request_id': 'req-54569807-7634-4b25-8292-b80cf081a0bb'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:44:00.104225 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-54569807-7634-4b25-8292-b80cf081a0bb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '42066c73ac8f4e5c883204dfc66245c0', 'request_id': 'req-54569807-7634-4b25-8292-b80cf081a0bb'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:44:00.111273 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-54569807-7634-4b25-8292-b80cf081a0bb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '42066c73ac8f4e5c883204dfc66245c0', 'request_id': 'req-54569807-7634-4b25-8292-b80cf081a0bb'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:44:00.124692 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-54569807-7634-4b25-8292-b80cf081a0bb admin admin] The token cache is being invalidate because project 42066c73ac8f4e5c883204dfc66245c0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80249) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 05:44:00.125086 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-54569807-7634-4b25-8292-b80cf081a0bb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:44:00.125691 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-54569807-7634-4b25-8292-b80cf081a0bb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:44:00.127253 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1237/2477] 10.222.0.33 () {58 vars in 1232 bytes} [Mon Jul 25 05:43:59 2022] DELETE /identity/v3/projects/42066c73ac8f4e5c883204dfc66245c0 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 05:44:00.132911 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1743656-639e-4595-8d64-a7f601d745d2 None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:44:00.133207 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1743656-639e-4595-8d64-a7f601d745d2 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:44:00.133487 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1743656-639e-4595-8d64-a7f601d745d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:44:00.150525 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-b1743656-639e-4595-8d64-a7f601d745d2 None None] MFA Rules not processed for user `1eae540242d94c10b0ad33960a855f08`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:44:00.151728 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-b1743656-639e-4595-8d64-a7f601d745d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:44:00.209278 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1241/2478] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 25 05:44:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:44:00.216853 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f4c46f-78f7-45b1-9ad7-5cc4f7a363a0 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:44:00.217161 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f4c46f-78f7-45b1-9ad7-5cc4f7a363a0 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:44:00.217425 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f4c46f-78f7-45b1-9ad7-5cc4f7a363a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:44:00.236141 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-11f4c46f-78f7-45b1-9ad7-5cc4f7a363a0 None None] MFA Rules not processed for user `1eae540242d94c10b0ad33960a855f08`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:44:00.237379 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-11f4c46f-78f7-45b1-9ad7-5cc4f7a363a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:44:00.246078 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1238/2479] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 25 05:44:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:44:00.250410 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1743656-639e-4595-8d64-a7f601d745d2 None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:44:00.260599 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1743656-639e-4595-8d64-a7f601d745d2 None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:44:00.265095 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdcb020a-6c82-4855-ba0d-c36f2cc96720 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:44:00.265745 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdcb020a-6c82-4855-ba0d-c36f2cc96720 admin admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:44:00.266026 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdcb020a-6c82-4855-ba0d-c36f2cc96720 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:44:00.266300 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdcb020a-6c82-4855-ba0d-c36f2cc96720 admin admin] PATH_INFO: `/v3/domains` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:44:00.267024 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdcb020a-6c82-4855-ba0d-c36f2cc96720 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:44:00.267330 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdcb020a-6c82-4855-ba0d-c36f2cc96720 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:44:00.268987 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdcb020a-6c82-4855-ba0d-c36f2cc96720 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:44:00.280701 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1242/2480] 10.222.0.33 () {58 vars in 1152 bytes} [Mon Jul 25 05:44:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:44:00.285000 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f4c46f-78f7-45b1-9ad7-5cc4f7a363a0 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:44:00.294200 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f4c46f-78f7-45b1-9ad7-5cc4f7a363a0 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:44:00.298726 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9beae5cf-cd83-4e7e-ba8c-fffb60d2f1fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:44:00.299370 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9beae5cf-cd83-4e7e-ba8c-fffb60d2f1fe admin admin] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:44:00.299650 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9beae5cf-cd83-4e7e-ba8c-fffb60d2f1fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:44:00.300456 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9beae5cf-cd83-4e7e-ba8c-fffb60d2f1fe admin admin] PATH_INFO: `/v3/projects` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:44:00.301206 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9beae5cf-cd83-4e7e-ba8c-fffb60d2f1fe admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:44:00.303089 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9beae5cf-cd83-4e7e-ba8c-fffb60d2f1fe admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:44:00.322532 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:44:00.332818 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1239/2481] 10.222.0.33 () {60 vars in 1152 bytes} [Mon Jul 25 05:44:00 2022] POST /identity/v3/projects => generated 375 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 05:44:00.334712 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdcb020a-6c82-4855-ba0d-c36f2cc96720 admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:44:00.344493 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdcb020a-6c82-4855-ba0d-c36f2cc96720 admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:44:00.388338 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:44:00.402489 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc121a9e-9e18-4cfa-83ae-fa77ed78b3dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:44:00.403156 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc121a9e-9e18-4cfa-83ae-fa77ed78b3dc admin admin] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:44:00.403450 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc121a9e-9e18-4cfa-83ae-fa77ed78b3dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:44:00.403744 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc121a9e-9e18-4cfa-83ae-fa77ed78b3dc admin admin] PATH_INFO: `/v3/users` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:44:00.404433 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc121a9e-9e18-4cfa-83ae-fa77ed78b3dc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:44:00.406419 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc121a9e-9e18-4cfa-83ae-fa77ed78b3dc admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:44:00.428288 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1243/2482] 10.222.0.33 () {60 vars in 1143 bytes} [Mon Jul 25 05:44:00 2022] POST /identity/v3/users => generated 288 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 05:44:00.432120 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9beae5cf-cd83-4e7e-ba8c-fffb60d2f1fe admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:44:00.444590 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9beae5cf-cd83-4e7e-ba8c-fffb60d2f1fe admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:44:00.449520 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e669d5-dc98-4191-b01f-c81225f89cd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:44:00.450182 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e669d5-dc98-4191-b01f-c81225f89cd6 admin admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:44:00.450466 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e669d5-dc98-4191-b01f-c81225f89cd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:44:00.450742 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e669d5-dc98-4191-b01f-c81225f89cd6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:44:00.451469 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7e669d5-dc98-4191-b01f-c81225f89cd6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:44:00.453078 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7e669d5-dc98-4191-b01f-c81225f89cd6 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:44:00.463498 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1240/2483] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jul 25 05:44:00 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 05:44:00.471215 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc121a9e-9e18-4cfa-83ae-fa77ed78b3dc admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:44:00.482142 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc121a9e-9e18-4cfa-83ae-fa77ed78b3dc admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:44:00.487008 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b73406-5a95-4897-ab37-898ce4a5e4e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:44:00.488233 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b73406-5a95-4897-ab37-898ce4a5e4e3 admin admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:44:00.488530 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b73406-5a95-4897-ab37-898ce4a5e4e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:44:00.488836 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b73406-5a95-4897-ab37-898ce4a5e4e3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:44:00.489640 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72b73406-5a95-4897-ab37-898ce4a5e4e3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:44:00.491240 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72b73406-5a95-4897-ab37-898ce4a5e4e3 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:44:00.501351 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1244/2484] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jul 25 05:44:00 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 05:44:00.505489 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e669d5-dc98-4191-b01f-c81225f89cd6 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:44:00.515356 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e669d5-dc98-4191-b01f-c81225f89cd6 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:44:00.519942 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f20f38-b8c0-4532-bb0c-2779cff860a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:44:00.520826 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f20f38-b8c0-4532-bb0c-2779cff860a3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:44:00.521113 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f20f38-b8c0-4532-bb0c-2779cff860a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:44:00.521389 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f20f38-b8c0-4532-bb0c-2779cff860a3 admin admin] PATH_INFO: `/v3/projects/16d64ce24f244cb5b4ba265faecf4d04/users/661e1ffef5d94740b16789823ba08d54/roles/5074e74e35f54d4b9bb415c34999a82a` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:44:00.535943 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3f20f38-b8c0-4532-bb0c-2779cff860a3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=16d64ce24f244cb5b4ba265faecf4d04, user_id=661e1ffef5d94740b16789823ba08d54, role_id=5074e74e35f54d4b9bb415c34999a82a)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:44:00.538630 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3f20f38-b8c0-4532-bb0c-2779cff860a3 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:44:00.552059 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1241/2485] 10.222.0.33 () {60 vars in 1482 bytes} [Mon Jul 25 05:44:00 2022] PUT /identity/v3/projects/16d64ce24f244cb5b4ba265faecf4d04/users/661e1ffef5d94740b16789823ba08d54/roles/5074e74e35f54d4b9bb415c34999a82a => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 05:44:00.562716 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041dd801-893d-4dff-acf7-0261af5d9946 None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:44:00.563012 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041dd801-893d-4dff-acf7-0261af5d9946 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:44:00.563301 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041dd801-893d-4dff-acf7-0261af5d9946 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:44:00.590220 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-041dd801-893d-4dff-acf7-0261af5d9946 None None] MFA Rules not processed for user `661e1ffef5d94740b16789823ba08d54`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:44:00.591441 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-041dd801-893d-4dff-acf7-0261af5d9946 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:44:00.661392 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1245/2486] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 25 05:44:00 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:44:00.672866 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f20f38-b8c0-4532-bb0c-2779cff860a3 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:44:00.674907 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-d3f20f38-b8c0-4532-bb0c-2779cff860a3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:44:00.749900 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f20f38-b8c0-4532-bb0c-2779cff860a3 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:44:00.756031 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2360af8-638d-4b9b-b62c-27565faa9953 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cd482a61e8a420db45bb532e5dae6e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92afba266cce4d1bad4134c351e74be3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:44:00.756682 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2360af8-638d-4b9b-b62c-27565faa9953 service nova] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:44:00.756996 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2360af8-638d-4b9b-b62c-27565faa9953 service nova] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:44:00.757272 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2360af8-638d-4b9b-b62c-27565faa9953 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:44:00.764939 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2360af8-638d-4b9b-b62c-27565faa9953 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:44:00.766511 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2360af8-638d-4b9b-b62c-27565faa9953 service nova] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:44:00.775722 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1242/2487] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jul 25 05:44:00 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 05:44:00.985507 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-041dd801-893d-4dff-acf7-0261af5d9946 None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:44:01.046476 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-041dd801-893d-4dff-acf7-0261af5d9946 None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:44:01.051630 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c4b6f3c-59a1-40d6-892d-7b26aab6f3c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:44:01.052545 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4b6f3c-59a1-40d6-892d-7b26aab6f3c7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:44:01.052800 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4b6f3c-59a1-40d6-892d-7b26aab6f3c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:44:01.053089 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4b6f3c-59a1-40d6-892d-7b26aab6f3c7 admin admin] PATH_INFO: `/v3/users/661e1ffef5d94740b16789823ba08d54` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:44:01.054276 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c4b6f3c-59a1-40d6-892d-7b26aab6f3c7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=661e1ffef5d94740b16789823ba08d54)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:44:01.056189 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c4b6f3c-59a1-40d6-892d-7b26aab6f3c7 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:44:01.104561 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-4c4b6f3c-59a1-40d6-892d-7b26aab6f3c7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '661e1ffef5d94740b16789823ba08d54'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:44:01.112707 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=89,keystone:INSERT=3,keystone:DELETE=8 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:44:01.119883 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-4c4b6f3c-59a1-40d6-892d-7b26aab6f3c7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '661e1ffef5d94740b16789823ba08d54'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:44:01.120644 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-4c4b6f3c-59a1-40d6-892d-7b26aab6f3c7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '661e1ffef5d94740b16789823ba08d54'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:44:01.128723 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-4c4b6f3c-59a1-40d6-892d-7b26aab6f3c7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '661e1ffef5d94740b16789823ba08d54'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:44:01.130663 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-4c4b6f3c-59a1-40d6-892d-7b26aab6f3c7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '661e1ffef5d94740b16789823ba08d54'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:44:01.144955 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1246/2488] 10.222.0.33 () {58 vars in 1223 bytes} [Mon Jul 25 05:44:00 2022] DELETE /identity/v3/users/661e1ffef5d94740b16789823ba08d54 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 05:44:01.149677 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0774437-e853-439c-b7ee-8b33a0e24c39 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:44:01.150193 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0774437-e853-439c-b7ee-8b33a0e24c39 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:44:01.150409 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0774437-e853-439c-b7ee-8b33a0e24c39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:44:01.168061 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-b0774437-e853-439c-b7ee-8b33a0e24c39 None None] MFA Rules not processed for user `1eae540242d94c10b0ad33960a855f08`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:44:01.169308 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-b0774437-e853-439c-b7ee-8b33a0e24c39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:44:01.230090 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1243/2489] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 25 05:44:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:44:01.241458 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c4b6f3c-59a1-40d6-892d-7b26aab6f3c7 admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:44:01.243489 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-4c4b6f3c-59a1-40d6-892d-7b26aab6f3c7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:44:01.301589 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c4b6f3c-59a1-40d6-892d-7b26aab6f3c7 admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:44:01.305376 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d257f24e-27f2-4652-931d-29116c3a0e37 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b2931200f99e43d79a50a9f6e02738e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92afba266cce4d1bad4134c351e74be3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:44:01.305964 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d257f24e-27f2-4652-931d-29116c3a0e37 service neutron] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:44:01.306252 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d257f24e-27f2-4652-931d-29116c3a0e37 service neutron] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:44:01.306531 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d257f24e-27f2-4652-931d-29116c3a0e37 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:44:01.315487 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d257f24e-27f2-4652-931d-29116c3a0e37 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:44:01.317119 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d257f24e-27f2-4652-931d-29116c3a0e37 service neutron] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:44:01.326437 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1247/2490] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jul 25 05:44:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 05:44:01.589299 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0774437-e853-439c-b7ee-8b33a0e24c39 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:44:01.590757 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-b0774437-e853-439c-b7ee-8b33a0e24c39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:44:01.603348 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0774437-e853-439c-b7ee-8b33a0e24c39 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:44:01.608204 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f33ca16b-a635-4d10-9fd1-fec33aed77de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:44:01.609120 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33ca16b-a635-4d10-9fd1-fec33aed77de admin admin] REQUEST_METHOD: `DELETE` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:44:01.609409 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33ca16b-a635-4d10-9fd1-fec33aed77de admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:44:01.609694 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33ca16b-a635-4d10-9fd1-fec33aed77de admin admin] PATH_INFO: `/v3/projects/16d64ce24f244cb5b4ba265faecf4d04` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:44:01.611039 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f33ca16b-a635-4d10-9fd1-fec33aed77de admin admin] RBAC: Authorizing `identity:delete_project(project_id=16d64ce24f244cb5b4ba265faecf4d04)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:44:01.613057 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f33ca16b-a635-4d10-9fd1-fec33aed77de admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:44:01.658418 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=96,keystone:INSERT=1,keystone:DELETE=3 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:44:01.661665 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-f33ca16b-a635-4d10-9fd1-fec33aed77de admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '16d64ce24f244cb5b4ba265faecf4d04', 'request_id': 'req-f33ca16b-a635-4d10-9fd1-fec33aed77de'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:44:01.668251 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-f33ca16b-a635-4d10-9fd1-fec33aed77de admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '16d64ce24f244cb5b4ba265faecf4d04', 'request_id': 'req-f33ca16b-a635-4d10-9fd1-fec33aed77de'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:44:01.676191 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-f33ca16b-a635-4d10-9fd1-fec33aed77de admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '16d64ce24f244cb5b4ba265faecf4d04', 'request_id': 'req-f33ca16b-a635-4d10-9fd1-fec33aed77de'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:44:01.688720 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-f33ca16b-a635-4d10-9fd1-fec33aed77de admin admin] The token cache is being invalidate because project 16d64ce24f244cb5b4ba265faecf4d04 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80249) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 05:44:01.689040 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-f33ca16b-a635-4d10-9fd1-fec33aed77de admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:44:01.689664 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-f33ca16b-a635-4d10-9fd1-fec33aed77de admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:44:01.691215 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1244/2491] 10.222.0.33 () {58 vars in 1232 bytes} [Mon Jul 25 05:44:01 2022] DELETE /identity/v3/projects/16d64ce24f244cb5b4ba265faecf4d04 => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 05:44:01.696312 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414d31bc-2826-4b88-9722-e70c5eef68ba None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:44:01.696677 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414d31bc-2826-4b88-9722-e70c5eef68ba None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:44:01.696995 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414d31bc-2826-4b88-9722-e70c5eef68ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:44:01.714687 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-414d31bc-2826-4b88-9722-e70c5eef68ba None None] MFA Rules not processed for user `1eae540242d94c10b0ad33960a855f08`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:44:01.715888 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-414d31bc-2826-4b88-9722-e70c5eef68ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:44:01.775103 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1248/2492] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 25 05:44:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:44:01.782884 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c51e9f-ef1e-49b6-9c50-944acffc3ae5 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:44:01.783180 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c51e9f-ef1e-49b6-9c50-944acffc3ae5 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:44:01.783464 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c51e9f-ef1e-49b6-9c50-944acffc3ae5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:44:01.802226 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-d8c51e9f-ef1e-49b6-9c50-944acffc3ae5 None None] MFA Rules not processed for user `1eae540242d94c10b0ad33960a855f08`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:44:01.803457 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-d8c51e9f-ef1e-49b6-9c50-944acffc3ae5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:44:01.811511 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1245/2493] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 25 05:44:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:44:01.815881 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-414d31bc-2826-4b88-9722-e70c5eef68ba None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:44:01.825757 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-414d31bc-2826-4b88-9722-e70c5eef68ba None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:44:01.830283 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af96895e-776d-4774-a8fe-d42dad46c4e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:44:01.830932 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af96895e-776d-4774-a8fe-d42dad46c4e3 admin admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:44:01.831221 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af96895e-776d-4774-a8fe-d42dad46c4e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:44:01.831500 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af96895e-776d-4774-a8fe-d42dad46c4e3 admin admin] PATH_INFO: `/v3/domains` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:44:01.832999 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af96895e-776d-4774-a8fe-d42dad46c4e3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:44:01.833262 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af96895e-776d-4774-a8fe-d42dad46c4e3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:44:01.834885 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af96895e-776d-4774-a8fe-d42dad46c4e3 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:44:01.846231 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1249/2494] 10.222.0.33 () {58 vars in 1152 bytes} [Mon Jul 25 05:44:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:44:01.850774 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c51e9f-ef1e-49b6-9c50-944acffc3ae5 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:44:01.860720 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c51e9f-ef1e-49b6-9c50-944acffc3ae5 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:44:01.867953 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0934bd50-c401-415c-b2f5-f38d4e0f37b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:44:01.868605 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0934bd50-c401-415c-b2f5-f38d4e0f37b9 admin admin] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:44:01.868925 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0934bd50-c401-415c-b2f5-f38d4e0f37b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:44:01.869203 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0934bd50-c401-415c-b2f5-f38d4e0f37b9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:44:01.869907 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0934bd50-c401-415c-b2f5-f38d4e0f37b9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:44:01.871807 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0934bd50-c401-415c-b2f5-f38d4e0f37b9 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:44:01.897862 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1246/2495] 10.222.0.33 () {60 vars in 1152 bytes} [Mon Jul 25 05:44:01 2022] POST /identity/v3/projects => generated 363 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 05:44:01.901652 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af96895e-776d-4774-a8fe-d42dad46c4e3 admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:44:01.912196 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af96895e-776d-4774-a8fe-d42dad46c4e3 admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:44:01.968336 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa71280d-e73b-4d5f-81ac-ab6d59b55d70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:44:01.969041 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa71280d-e73b-4d5f-81ac-ab6d59b55d70 admin admin] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:44:01.969328 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa71280d-e73b-4d5f-81ac-ab6d59b55d70 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:44:01.969608 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa71280d-e73b-4d5f-81ac-ab6d59b55d70 admin admin] PATH_INFO: `/v3/users` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:44:01.970306 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa71280d-e73b-4d5f-81ac-ab6d59b55d70 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:44:01.972499 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa71280d-e73b-4d5f-81ac-ab6d59b55d70 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:44:01.981575 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=95,keystone:DELETE=2,keystone:INSERT=3 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:44:01.997966 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1250/2496] 10.222.0.33 () {60 vars in 1143 bytes} [Mon Jul 25 05:44:01 2022] POST /identity/v3/users => generated 282 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 05:44:02.002314 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0934bd50-c401-415c-b2f5-f38d4e0f37b9 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:44:02.012704 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0934bd50-c401-415c-b2f5-f38d4e0f37b9 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:44:02.017290 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-896a5c77-2c99-4715-ac95-9e9d13961c50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:44:02.017950 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896a5c77-2c99-4715-ac95-9e9d13961c50 admin admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:44:02.018232 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896a5c77-2c99-4715-ac95-9e9d13961c50 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:44:02.018507 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896a5c77-2c99-4715-ac95-9e9d13961c50 admin admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:44:02.019231 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-896a5c77-2c99-4715-ac95-9e9d13961c50 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:44:02.020839 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-896a5c77-2c99-4715-ac95-9e9d13961c50 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:44:02.031349 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1247/2497] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jul 25 05:44:02 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 05:44:02.036270 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa71280d-e73b-4d5f-81ac-ab6d59b55d70 admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:44:02.046407 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa71280d-e73b-4d5f-81ac-ab6d59b55d70 admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:44:02.051771 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e3f5ae-c2a6-437b-844a-d9942924a189 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:44:02.052487 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e3f5ae-c2a6-437b-844a-d9942924a189 admin admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:44:02.052974 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e3f5ae-c2a6-437b-844a-d9942924a189 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:44:02.053260 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e3f5ae-c2a6-437b-844a-d9942924a189 admin admin] PATH_INFO: `/v3/roles` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:44:02.054000 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82e3f5ae-c2a6-437b-844a-d9942924a189 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:44:02.055595 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82e3f5ae-c2a6-437b-844a-d9942924a189 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:44:02.066229 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1251/2498] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jul 25 05:44:02 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 05:44:02.071007 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-896a5c77-2c99-4715-ac95-9e9d13961c50 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:44:02.081090 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-896a5c77-2c99-4715-ac95-9e9d13961c50 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:44:02.086528 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69649ad8-be24-491b-a44e-541eb266317a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:44:02.087334 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69649ad8-be24-491b-a44e-541eb266317a admin admin] REQUEST_METHOD: `PUT` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:44:02.087615 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69649ad8-be24-491b-a44e-541eb266317a admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:44:02.087891 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69649ad8-be24-491b-a44e-541eb266317a admin admin] PATH_INFO: `/v3/projects/e0512788983949119cb33785d564d2b2/users/c838705668b647d295927440e09e7395/roles/5074e74e35f54d4b9bb415c34999a82a` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:44:02.102734 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69649ad8-be24-491b-a44e-541eb266317a admin admin] RBAC: Authorizing `identity:create_grant(project_id=e0512788983949119cb33785d564d2b2, user_id=c838705668b647d295927440e09e7395, role_id=5074e74e35f54d4b9bb415c34999a82a)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:44:02.106984 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69649ad8-be24-491b-a44e-541eb266317a admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:44:02.122913 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1248/2499] 10.222.0.33 () {60 vars in 1482 bytes} [Mon Jul 25 05:44:02 2022] PUT /identity/v3/projects/e0512788983949119cb33785d564d2b2/users/c838705668b647d295927440e09e7395/roles/5074e74e35f54d4b9bb415c34999a82a => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 05:44:02.134060 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803cf60b-2cde-4df4-b62b-e1f164d97284 None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:44:02.134371 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803cf60b-2cde-4df4-b62b-e1f164d97284 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:44:02.134657 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803cf60b-2cde-4df4-b62b-e1f164d97284 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:44:02.161188 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-803cf60b-2cde-4df4-b62b-e1f164d97284 None None] MFA Rules not processed for user `c838705668b647d295927440e09e7395`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:44:02.162405 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-803cf60b-2cde-4df4-b62b-e1f164d97284 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:44:02.226914 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1252/2500] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 25 05:44:02 2022] POST /identity/v3/auth/tokens => generated 3230 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:44:02.231396 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69649ad8-be24-491b-a44e-541eb266317a admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:44:02.294597 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69649ad8-be24-491b-a44e-541eb266317a admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:44:02.299223 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc7d3bd-9287-4d16-a4f0-7dc1f21c58ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:44:02.299884 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc7d3bd-9287-4d16-a4f0-7dc1f21c58ce admin admin] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:44:02.300168 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc7d3bd-9287-4d16-a4f0-7dc1f21c58ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:44:02.300444 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc7d3bd-9287-4d16-a4f0-7dc1f21c58ce admin admin] PATH_INFO: `/v3/projects` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:44:02.301754 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbc7d3bd-9287-4d16-a4f0-7dc1f21c58ce admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:44:02.303762 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbc7d3bd-9287-4d16-a4f0-7dc1f21c58ce admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:44:02.333213 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1249/2501] 10.222.0.33 () {60 vars in 1152 bytes} [Mon Jul 25 05:44:02 2022] POST /identity/v3/projects => generated 363 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 05:44:02.334517 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-803cf60b-2cde-4df4-b62b-e1f164d97284 None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:44:02.345090 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-803cf60b-2cde-4df4-b62b-e1f164d97284 None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:44:02.407832 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a2b7536-68a8-4a3d-b550-5e020c60f995 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:44:02.408631 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2b7536-68a8-4a3d-b550-5e020c60f995 admin admin] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:44:02.409020 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2b7536-68a8-4a3d-b550-5e020c60f995 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:44:02.409303 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2b7536-68a8-4a3d-b550-5e020c60f995 admin admin] PATH_INFO: `/v3/users` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:44:02.410000 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a2b7536-68a8-4a3d-b550-5e020c60f995 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:44:02.412063 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a2b7536-68a8-4a3d-b550-5e020c60f995 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:44:02.437071 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1253/2502] 10.222.0.33 () {60 vars in 1143 bytes} [Mon Jul 25 05:44:02 2022] POST /identity/v3/users => generated 288 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 05:44:02.438220 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc7d3bd-9287-4d16-a4f0-7dc1f21c58ce admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:44:02.449940 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc7d3bd-9287-4d16-a4f0-7dc1f21c58ce admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:44:02.454691 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-474818c2-c220-4bf0-aa9c-55c0f887ca3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:44:02.455359 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474818c2-c220-4bf0-aa9c-55c0f887ca3e admin admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:44:02.455647 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474818c2-c220-4bf0-aa9c-55c0f887ca3e admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:44:02.455929 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474818c2-c220-4bf0-aa9c-55c0f887ca3e admin admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:44:02.456664 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-474818c2-c220-4bf0-aa9c-55c0f887ca3e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:44:02.459075 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-474818c2-c220-4bf0-aa9c-55c0f887ca3e admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:44:02.469896 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1250/2503] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jul 25 05:44:02 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 25 05:44:02.475033 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a2b7536-68a8-4a3d-b550-5e020c60f995 admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:44:02.484896 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a2b7536-68a8-4a3d-b550-5e020c60f995 admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:44:02.490040 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24809600-ffd5-46cd-9f7f-1fb0335e5ec0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:44:02.490908 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24809600-ffd5-46cd-9f7f-1fb0335e5ec0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:44:02.491192 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24809600-ffd5-46cd-9f7f-1fb0335e5ec0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:44:02.491467 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24809600-ffd5-46cd-9f7f-1fb0335e5ec0 admin admin] PATH_INFO: `/v3/projects/c80d05fe950a4e138b4dedcf71d6e47c/users/0980f840bfa4444aab8639efe4692f1d/roles/39e5c666285a42fe8fc2c3afca6260d1` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:44:02.506590 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24809600-ffd5-46cd-9f7f-1fb0335e5ec0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c80d05fe950a4e138b4dedcf71d6e47c, user_id=0980f840bfa4444aab8639efe4692f1d, role_id=39e5c666285a42fe8fc2c3afca6260d1)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:44:02.509264 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24809600-ffd5-46cd-9f7f-1fb0335e5ec0 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:44:02.521957 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1254/2504] 10.222.0.33 () {60 vars in 1482 bytes} [Mon Jul 25 05:44:02 2022] PUT /identity/v3/projects/c80d05fe950a4e138b4dedcf71d6e47c/users/0980f840bfa4444aab8639efe4692f1d/roles/39e5c666285a42fe8fc2c3afca6260d1 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 05:44:02.525925 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-474818c2-c220-4bf0-aa9c-55c0f887ca3e admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:44:02.535946 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-474818c2-c220-4bf0-aa9c-55c0f887ca3e admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:44:02.577043 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:DELETE=4,keystone:SELECT=92,keystone:INSERT=4 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:44:02.594040 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce48610b-fd58-46ac-8679-5ad946af5b37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:44:02.594749 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce48610b-fd58-46ac-8679-5ad946af5b37 admin admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:44:02.595037 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce48610b-fd58-46ac-8679-5ad946af5b37 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:44:02.595361 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce48610b-fd58-46ac-8679-5ad946af5b37 admin admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:44:02.596042 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce48610b-fd58-46ac-8679-5ad946af5b37 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:44:02.598079 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce48610b-fd58-46ac-8679-5ad946af5b37 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:44:02.607805 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1251/2505] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jul 25 05:44:02 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 05:44:02.611934 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24809600-ffd5-46cd-9f7f-1fb0335e5ec0 admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:44:02.622917 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24809600-ffd5-46cd-9f7f-1fb0335e5ec0 admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:44:02.627524 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f508db0d-1f8b-4312-a529-63e8574a27e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:44:02.628393 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f508db0d-1f8b-4312-a529-63e8574a27e7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:44:02.628673 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f508db0d-1f8b-4312-a529-63e8574a27e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:44:02.629133 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f508db0d-1f8b-4312-a529-63e8574a27e7 admin admin] PATH_INFO: `/v3/projects/c80d05fe950a4e138b4dedcf71d6e47c/users/0980f840bfa4444aab8639efe4692f1d/roles/39e5c666285a42fe8fc2c3afca6260d1` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:44:02.631184 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f508db0d-1f8b-4312-a529-63e8574a27e7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c80d05fe950a4e138b4dedcf71d6e47c, user_id=0980f840bfa4444aab8639efe4692f1d, role_id=39e5c666285a42fe8fc2c3afca6260d1)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:44:02.633830 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f508db0d-1f8b-4312-a529-63e8574a27e7 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:44:02.643832 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1255/2506] 10.222.0.33 () {60 vars in 1482 bytes} [Mon Jul 25 05:44:02 2022] PUT /identity/v3/projects/c80d05fe950a4e138b4dedcf71d6e47c/users/0980f840bfa4444aab8639efe4692f1d/roles/39e5c666285a42fe8fc2c3afca6260d1 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 05:44:02.656207 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27353eef-518f-42dc-b25d-fbd1db513e4c None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:44:02.656499 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27353eef-518f-42dc-b25d-fbd1db513e4c None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:44:02.656796 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27353eef-518f-42dc-b25d-fbd1db513e4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:44:02.684945 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-27353eef-518f-42dc-b25d-fbd1db513e4c None None] MFA Rules not processed for user `0980f840bfa4444aab8639efe4692f1d`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:44:02.686170 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-27353eef-518f-42dc-b25d-fbd1db513e4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:44:02.754240 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1252/2507] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 25 05:44:02 2022] POST /identity/v3/auth/tokens => generated 3297 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:44:02.765014 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f508db0d-1f8b-4312-a529-63e8574a27e7 admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:44:02.766749 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-f508db0d-1f8b-4312-a529-63e8574a27e7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:44:02.794551 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:INSERT=6,keystone:SELECT=94 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:44:02.825119 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f508db0d-1f8b-4312-a529-63e8574a27e7 admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:44:02.828727 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee7aa672-538a-4a2a-b3e5-35229bca4eaf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db57b478c25643dbaf936f90f6e17d0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92afba266cce4d1bad4134c351e74be3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:44:02.829390 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7aa672-538a-4a2a-b3e5-35229bca4eaf service cinder] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:44:02.829672 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7aa672-538a-4a2a-b3e5-35229bca4eaf service cinder] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:44:02.829946 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7aa672-538a-4a2a-b3e5-35229bca4eaf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:44:02.838856 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee7aa672-538a-4a2a-b3e5-35229bca4eaf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:44:02.840270 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee7aa672-538a-4a2a-b3e5-35229bca4eaf service cinder] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:44:02.850368 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1256/2508] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jul 25 05:44:02 2022] GET /identity/v3/auth/tokens => generated 3297 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 05:44:03.257518 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27353eef-518f-42dc-b25d-fbd1db513e4c None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:44:03.319052 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27353eef-518f-42dc-b25d-fbd1db513e4c None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:44:03.323471 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3202651-6b95-4077-a91b-25644fbc18d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:44:03.324356 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3202651-6b95-4077-a91b-25644fbc18d0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:44:03.324637 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3202651-6b95-4077-a91b-25644fbc18d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:44:03.324938 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3202651-6b95-4077-a91b-25644fbc18d0 admin admin] PATH_INFO: `/v3/users/c838705668b647d295927440e09e7395` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:44:03.326109 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3202651-6b95-4077-a91b-25644fbc18d0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c838705668b647d295927440e09e7395)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:44:03.328020 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3202651-6b95-4077-a91b-25644fbc18d0 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:44:03.371902 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-d3202651-6b95-4077-a91b-25644fbc18d0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c838705668b647d295927440e09e7395'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:44:03.372473 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-d3202651-6b95-4077-a91b-25644fbc18d0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c838705668b647d295927440e09e7395'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:44:03.379782 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-d3202651-6b95-4077-a91b-25644fbc18d0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c838705668b647d295927440e09e7395'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:44:03.387358 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=92,keystone:DELETE=8 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:44:03.392463 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-d3202651-6b95-4077-a91b-25644fbc18d0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c838705668b647d295927440e09e7395'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:44:03.393273 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-d3202651-6b95-4077-a91b-25644fbc18d0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c838705668b647d295927440e09e7395'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:44:03.406215 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1253/2509] 10.222.0.33 () {58 vars in 1223 bytes} [Mon Jul 25 05:44:03 2022] DELETE /identity/v3/users/c838705668b647d295927440e09e7395 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 05:44:03.410909 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd699907-9294-4420-88e9-47356eba9007 None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:44:03.411420 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd699907-9294-4420-88e9-47356eba9007 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:44:03.411522 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd699907-9294-4420-88e9-47356eba9007 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:44:03.429500 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-bd699907-9294-4420-88e9-47356eba9007 None None] MFA Rules not processed for user `1eae540242d94c10b0ad33960a855f08`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:44:03.430721 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-bd699907-9294-4420-88e9-47356eba9007 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:44:03.493389 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1257/2510] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 25 05:44:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:44:03.504206 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3202651-6b95-4077-a91b-25644fbc18d0 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:44:03.506334 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-d3202651-6b95-4077-a91b-25644fbc18d0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:44:03.569553 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3202651-6b95-4077-a91b-25644fbc18d0 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:44:03.573248 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3679d8e5-b275-4004-9a9d-b536e4a679eb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b2931200f99e43d79a50a9f6e02738e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92afba266cce4d1bad4134c351e74be3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:44:03.573899 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3679d8e5-b275-4004-9a9d-b536e4a679eb service neutron] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:44:03.574181 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3679d8e5-b275-4004-9a9d-b536e4a679eb service neutron] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:44:03.574457 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3679d8e5-b275-4004-9a9d-b536e4a679eb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:44:03.584129 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3679d8e5-b275-4004-9a9d-b536e4a679eb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:44:03.585619 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3679d8e5-b275-4004-9a9d-b536e4a679eb service neutron] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:44:03.598240 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1254/2511] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jul 25 05:44:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 05:44:03.893700 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd699907-9294-4420-88e9-47356eba9007 None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:44:03.895828 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-bd699907-9294-4420-88e9-47356eba9007 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:44:03.908866 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd699907-9294-4420-88e9-47356eba9007 None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:44:03.914091 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cdb6b54-43b1-4e53-bf4e-e35dad7ce08a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:44:03.915627 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cdb6b54-43b1-4e53-bf4e-e35dad7ce08a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:44:03.916241 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cdb6b54-43b1-4e53-bf4e-e35dad7ce08a admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:44:03.916552 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cdb6b54-43b1-4e53-bf4e-e35dad7ce08a admin admin] PATH_INFO: `/v3/projects/e0512788983949119cb33785d564d2b2` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:44:03.917804 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cdb6b54-43b1-4e53-bf4e-e35dad7ce08a admin admin] RBAC: Authorizing `identity:delete_project(project_id=e0512788983949119cb33785d564d2b2)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:44:03.919696 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cdb6b54-43b1-4e53-bf4e-e35dad7ce08a admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:44:03.967091 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-7cdb6b54-43b1-4e53-bf4e-e35dad7ce08a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e0512788983949119cb33785d564d2b2', 'request_id': 'req-7cdb6b54-43b1-4e53-bf4e-e35dad7ce08a'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:44:03.973541 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-7cdb6b54-43b1-4e53-bf4e-e35dad7ce08a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e0512788983949119cb33785d564d2b2', 'request_id': 'req-7cdb6b54-43b1-4e53-bf4e-e35dad7ce08a'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:44:03.979685 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-7cdb6b54-43b1-4e53-bf4e-e35dad7ce08a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e0512788983949119cb33785d564d2b2', 'request_id': 'req-7cdb6b54-43b1-4e53-bf4e-e35dad7ce08a'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:44:03.991898 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-7cdb6b54-43b1-4e53-bf4e-e35dad7ce08a admin admin] The token cache is being invalidate because project e0512788983949119cb33785d564d2b2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80248) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 05:44:03.992183 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-7cdb6b54-43b1-4e53-bf4e-e35dad7ce08a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:44:03.992821 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-7cdb6b54-43b1-4e53-bf4e-e35dad7ce08a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:44:03.994391 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1258/2512] 10.222.0.33 () {58 vars in 1232 bytes} [Mon Jul 25 05:44:03 2022] DELETE /identity/v3/projects/e0512788983949119cb33785d564d2b2 => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 05:44:03.998541 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3679d8e5-b275-4004-9a9d-b536e4a679eb service neutron] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:44:04.008227 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3679d8e5-b275-4004-9a9d-b536e4a679eb service neutron] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:44:04.064971 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7400f58-0540-4f0f-bdbd-993a3773859b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:44:04.065826 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7400f58-0540-4f0f-bdbd-993a3773859b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:44:04.066109 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7400f58-0540-4f0f-bdbd-993a3773859b admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:44:04.066953 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7400f58-0540-4f0f-bdbd-993a3773859b admin admin] PATH_INFO: `/v3/users/0980f840bfa4444aab8639efe4692f1d` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:44:04.068110 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7400f58-0540-4f0f-bdbd-993a3773859b admin admin] RBAC: Authorizing `identity:delete_user(user_id=0980f840bfa4444aab8639efe4692f1d)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:44:04.070059 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7400f58-0540-4f0f-bdbd-993a3773859b admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:44:04.116052 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-f7400f58-0540-4f0f-bdbd-993a3773859b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0980f840bfa4444aab8639efe4692f1d'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:44:04.116717 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-f7400f58-0540-4f0f-bdbd-993a3773859b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0980f840bfa4444aab8639efe4692f1d'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:44:04.125306 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-f7400f58-0540-4f0f-bdbd-993a3773859b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0980f840bfa4444aab8639efe4692f1d'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:44:04.140918 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-f7400f58-0540-4f0f-bdbd-993a3773859b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0980f840bfa4444aab8639efe4692f1d'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:44:04.141682 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-f7400f58-0540-4f0f-bdbd-993a3773859b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0980f840bfa4444aab8639efe4692f1d'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:44:04.154984 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1255/2513] 10.222.0.33 () {58 vars in 1223 bytes} [Mon Jul 25 05:44:03 2022] DELETE /identity/v3/users/0980f840bfa4444aab8639efe4692f1d => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 05:44:04.440113 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cdb6b54-43b1-4e53-bf4e-e35dad7ce08a admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:44:04.441721 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-7cdb6b54-43b1-4e53-bf4e-e35dad7ce08a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:44:04.454734 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cdb6b54-43b1-4e53-bf4e-e35dad7ce08a admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:44:04.503949 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:44:04.515681 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac29edd-978c-475e-8f38-0aaf6d98e465 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:44:04.516541 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac29edd-978c-475e-8f38-0aaf6d98e465 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:44:04.516864 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac29edd-978c-475e-8f38-0aaf6d98e465 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:44:04.517145 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac29edd-978c-475e-8f38-0aaf6d98e465 admin admin] PATH_INFO: `/v3/projects/c80d05fe950a4e138b4dedcf71d6e47c` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:44:04.518335 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ac29edd-978c-475e-8f38-0aaf6d98e465 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c80d05fe950a4e138b4dedcf71d6e47c)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:44:04.520387 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ac29edd-978c-475e-8f38-0aaf6d98e465 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:44:04.567836 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-9ac29edd-978c-475e-8f38-0aaf6d98e465 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c80d05fe950a4e138b4dedcf71d6e47c', 'request_id': 'req-9ac29edd-978c-475e-8f38-0aaf6d98e465'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:44:04.573741 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-9ac29edd-978c-475e-8f38-0aaf6d98e465 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c80d05fe950a4e138b4dedcf71d6e47c', 'request_id': 'req-9ac29edd-978c-475e-8f38-0aaf6d98e465'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:44:04.579882 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-9ac29edd-978c-475e-8f38-0aaf6d98e465 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c80d05fe950a4e138b4dedcf71d6e47c', 'request_id': 'req-9ac29edd-978c-475e-8f38-0aaf6d98e465'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:44:04.592979 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-9ac29edd-978c-475e-8f38-0aaf6d98e465 admin admin] The token cache is being invalidate because project c80d05fe950a4e138b4dedcf71d6e47c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80248) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 05:44:04.593272 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-9ac29edd-978c-475e-8f38-0aaf6d98e465 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:44:04.593900 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-9ac29edd-978c-475e-8f38-0aaf6d98e465 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:44:04.595503 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1259/2514] 10.222.0.33 () {58 vars in 1232 bytes} [Mon Jul 25 05:44:04 2022] DELETE /identity/v3/projects/c80d05fe950a4e138b4dedcf71d6e47c => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 05:44:04.601495 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755023dc-e55e-4d96-b398-90a9d414dbaf None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:44:04.601966 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755023dc-e55e-4d96-b398-90a9d414dbaf None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:44:04.602078 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755023dc-e55e-4d96-b398-90a9d414dbaf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:44:04.614721 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:44:04.621793 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-755023dc-e55e-4d96-b398-90a9d414dbaf None None] MFA Rules not processed for user `1eae540242d94c10b0ad33960a855f08`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:44:04.623684 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-755023dc-e55e-4d96-b398-90a9d414dbaf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:44:04.694793 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1256/2515] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 25 05:44:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:44:04.702668 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5d2df0-7ccf-4b4c-acf2-f1ea71403e11 None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:44:04.703043 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5d2df0-7ccf-4b4c-acf2-f1ea71403e11 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:44:04.703401 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5d2df0-7ccf-4b4c-acf2-f1ea71403e11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:44:04.722725 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-fa5d2df0-7ccf-4b4c-acf2-f1ea71403e11 None None] MFA Rules not processed for user `1eae540242d94c10b0ad33960a855f08`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:44:04.724637 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-fa5d2df0-7ccf-4b4c-acf2-f1ea71403e11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:44:04.732807 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1260/2516] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 25 05:44:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:44:04.737116 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-755023dc-e55e-4d96-b398-90a9d414dbaf None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:44:04.747861 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-755023dc-e55e-4d96-b398-90a9d414dbaf None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:44:04.752505 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23aa016f-6a95-4332-8f83-bba8fbdfbd81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:44:04.753211 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23aa016f-6a95-4332-8f83-bba8fbdfbd81 admin admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:44:04.753506 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23aa016f-6a95-4332-8f83-bba8fbdfbd81 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:44:04.753788 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23aa016f-6a95-4332-8f83-bba8fbdfbd81 admin admin] PATH_INFO: `/v3/domains` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:44:04.754565 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23aa016f-6a95-4332-8f83-bba8fbdfbd81 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:44:04.755385 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23aa016f-6a95-4332-8f83-bba8fbdfbd81 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:44:04.756503 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23aa016f-6a95-4332-8f83-bba8fbdfbd81 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:44:04.768913 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1257/2517] 10.222.0.33 () {58 vars in 1152 bytes} [Mon Jul 25 05:44:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:44:04.773656 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa5d2df0-7ccf-4b4c-acf2-f1ea71403e11 None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:44:04.783117 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa5d2df0-7ccf-4b4c-acf2-f1ea71403e11 None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:44:04.788222 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec701ee-af0e-4103-a21e-913119f123dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:44:04.788916 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec701ee-af0e-4103-a21e-913119f123dd admin admin] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:44:04.789213 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec701ee-af0e-4103-a21e-913119f123dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:44:04.789491 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec701ee-af0e-4103-a21e-913119f123dd admin admin] PATH_INFO: `/v3/projects` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:44:04.790187 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ec701ee-af0e-4103-a21e-913119f123dd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:44:04.792118 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ec701ee-af0e-4103-a21e-913119f123dd admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:44:04.822345 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1261/2518] 10.222.0.33 () {60 vars in 1152 bytes} [Mon Jul 25 05:44:04 2022] POST /identity/v3/projects => generated 357 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 05:44:04.826596 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23aa016f-6a95-4332-8f83-bba8fbdfbd81 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:44:04.837243 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23aa016f-6a95-4332-8f83-bba8fbdfbd81 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:44:04.895668 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06bbd4f0-8f91-4c3f-a12e-74ab3fd25f16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:44:04.896337 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06bbd4f0-8f91-4c3f-a12e-74ab3fd25f16 admin admin] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:44:04.896627 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06bbd4f0-8f91-4c3f-a12e-74ab3fd25f16 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:44:04.896950 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06bbd4f0-8f91-4c3f-a12e-74ab3fd25f16 admin admin] PATH_INFO: `/v3/users` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:44:04.897640 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06bbd4f0-8f91-4c3f-a12e-74ab3fd25f16 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:44:04.899566 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06bbd4f0-8f91-4c3f-a12e-74ab3fd25f16 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:44:04.918690 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1258/2519] 10.222.0.33 () {60 vars in 1143 bytes} [Mon Jul 25 05:44:04 2022] POST /identity/v3/users => generated 279 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 05:44:04.922676 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec701ee-af0e-4103-a21e-913119f123dd admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:44:04.934084 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec701ee-af0e-4103-a21e-913119f123dd admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:44:04.938726 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21230f97-47b9-4dd8-99a7-352cb0f7fcce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:44:04.939327 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21230f97-47b9-4dd8-99a7-352cb0f7fcce admin admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:44:04.939614 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21230f97-47b9-4dd8-99a7-352cb0f7fcce admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:44:04.939892 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21230f97-47b9-4dd8-99a7-352cb0f7fcce admin admin] PATH_INFO: `/v3/roles` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:44:04.940620 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21230f97-47b9-4dd8-99a7-352cb0f7fcce admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:44:04.943224 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21230f97-47b9-4dd8-99a7-352cb0f7fcce admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:44:04.953481 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1262/2520] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jul 25 05:44:04 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 05:44:04.957395 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06bbd4f0-8f91-4c3f-a12e-74ab3fd25f16 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:44:04.967446 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06bbd4f0-8f91-4c3f-a12e-74ab3fd25f16 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:44:04.972030 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-354f0baf-ea12-4a38-adc8-30b56bf5e499 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:44:04.973536 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354f0baf-ea12-4a38-adc8-30b56bf5e499 admin admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:44:04.973828 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354f0baf-ea12-4a38-adc8-30b56bf5e499 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:44:04.974222 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354f0baf-ea12-4a38-adc8-30b56bf5e499 admin admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:44:04.974964 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-354f0baf-ea12-4a38-adc8-30b56bf5e499 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:44:04.976551 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-354f0baf-ea12-4a38-adc8-30b56bf5e499 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:44:04.986475 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1259/2521] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jul 25 05:44:04 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 05:44:04.991460 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21230f97-47b9-4dd8-99a7-352cb0f7fcce admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:44:05.001453 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21230f97-47b9-4dd8-99a7-352cb0f7fcce admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:44:05.006250 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33bb113d-901a-4589-9de8-e40407e7a1ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:44:05.007145 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33bb113d-901a-4589-9de8-e40407e7a1ae admin admin] REQUEST_METHOD: `PUT` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:44:05.007436 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33bb113d-901a-4589-9de8-e40407e7a1ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:44:05.007716 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33bb113d-901a-4589-9de8-e40407e7a1ae admin admin] PATH_INFO: `/v3/projects/f501377f72e843e7859efbb51e7a47f5/users/6d0fcbedcff142a98c43f39e6a7ab0e4/roles/5074e74e35f54d4b9bb415c34999a82a` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:44:05.022411 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33bb113d-901a-4589-9de8-e40407e7a1ae admin admin] RBAC: Authorizing `identity:create_grant(project_id=f501377f72e843e7859efbb51e7a47f5, user_id=6d0fcbedcff142a98c43f39e6a7ab0e4, role_id=5074e74e35f54d4b9bb415c34999a82a)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:44:05.025107 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33bb113d-901a-4589-9de8-e40407e7a1ae admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:44:05.038655 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1263/2522] 10.222.0.33 () {60 vars in 1482 bytes} [Mon Jul 25 05:44:04 2022] PUT /identity/v3/projects/f501377f72e843e7859efbb51e7a47f5/users/6d0fcbedcff142a98c43f39e6a7ab0e4/roles/5074e74e35f54d4b9bb415c34999a82a => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 05:44:05.050046 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2d40cc-9bee-452c-9fc9-25c339d6ebd3 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:44:05.050341 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2d40cc-9bee-452c-9fc9-25c339d6ebd3 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:44:05.050623 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2d40cc-9bee-452c-9fc9-25c339d6ebd3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:44:05.076746 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-2b2d40cc-9bee-452c-9fc9-25c339d6ebd3 None None] MFA Rules not processed for user `6d0fcbedcff142a98c43f39e6a7ab0e4`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:44:05.077987 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-2b2d40cc-9bee-452c-9fc9-25c339d6ebd3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:44:05.141084 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1260/2523] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 25 05:44:05 2022] POST /identity/v3/auth/tokens => generated 3224 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:44:05.145992 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33bb113d-901a-4589-9de8-e40407e7a1ae admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:44:05.208576 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33bb113d-901a-4589-9de8-e40407e7a1ae admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:44:05.213179 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b584233-3207-4646-af06-6c8347b53e1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:44:05.213837 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b584233-3207-4646-af06-6c8347b53e1b admin admin] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:44:05.214126 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b584233-3207-4646-af06-6c8347b53e1b admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:44:05.214401 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b584233-3207-4646-af06-6c8347b53e1b admin admin] PATH_INFO: `/v3/projects` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:44:05.215105 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b584233-3207-4646-af06-6c8347b53e1b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:44:05.217068 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b584233-3207-4646-af06-6c8347b53e1b admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:44:05.243018 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1264/2524] 10.222.0.33 () {60 vars in 1152 bytes} [Mon Jul 25 05:44:05 2022] POST /identity/v3/projects => generated 355 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 25 05:44:05.247259 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b2d40cc-9bee-452c-9fc9-25c339d6ebd3 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:44:05.252708 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:44:05.259618 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b2d40cc-9bee-452c-9fc9-25c339d6ebd3 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:44:05.314642 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-846420ad-24f9-40f0-90f7-3477a32d835b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:44:05.315309 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846420ad-24f9-40f0-90f7-3477a32d835b admin admin] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:44:05.315597 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846420ad-24f9-40f0-90f7-3477a32d835b admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:44:05.315874 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846420ad-24f9-40f0-90f7-3477a32d835b admin admin] PATH_INFO: `/v3/users` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:44:05.316591 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-846420ad-24f9-40f0-90f7-3477a32d835b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:44:05.318539 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-846420ad-24f9-40f0-90f7-3477a32d835b admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:44:05.337884 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1261/2525] 10.222.0.33 () {60 vars in 1143 bytes} [Mon Jul 25 05:44:05 2022] POST /identity/v3/users => generated 284 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 25 05:44:05.342131 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b584233-3207-4646-af06-6c8347b53e1b admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:44:05.348037 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:44:05.354717 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b584233-3207-4646-af06-6c8347b53e1b admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:44:05.360380 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-359601fc-3e9a-4f68-9926-a002ea602b41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:44:05.361101 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359601fc-3e9a-4f68-9926-a002ea602b41 admin admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:44:05.361399 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359601fc-3e9a-4f68-9926-a002ea602b41 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:44:05.361682 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359601fc-3e9a-4f68-9926-a002ea602b41 admin admin] PATH_INFO: `/v3/roles` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:44:05.362417 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-359601fc-3e9a-4f68-9926-a002ea602b41 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:44:05.364720 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-359601fc-3e9a-4f68-9926-a002ea602b41 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:44:05.375297 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1265/2526] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jul 25 05:44:05 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 05:44:05.379517 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-846420ad-24f9-40f0-90f7-3477a32d835b admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:44:05.389469 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-846420ad-24f9-40f0-90f7-3477a32d835b admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:44:05.394120 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ff7569-0f0b-44c8-94dc-60dbf424b6dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:44:05.395527 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ff7569-0f0b-44c8-94dc-60dbf424b6dc admin admin] REQUEST_METHOD: `PUT` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:44:05.395807 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ff7569-0f0b-44c8-94dc-60dbf424b6dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:44:05.396116 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ff7569-0f0b-44c8-94dc-60dbf424b6dc admin admin] PATH_INFO: `/v3/projects/38a78d799aac4d4ab4c9c25f2ad5d930/users/835091aac8724ca5990bd892d68de552/roles/39e5c666285a42fe8fc2c3afca6260d1` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:44:05.411275 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8ff7569-0f0b-44c8-94dc-60dbf424b6dc admin admin] RBAC: Authorizing `identity:create_grant(project_id=38a78d799aac4d4ab4c9c25f2ad5d930, user_id=835091aac8724ca5990bd892d68de552, role_id=39e5c666285a42fe8fc2c3afca6260d1)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:44:05.413969 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8ff7569-0f0b-44c8-94dc-60dbf424b6dc admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:44:05.426427 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1262/2527] 10.222.0.33 () {60 vars in 1482 bytes} [Mon Jul 25 05:44:05 2022] PUT /identity/v3/projects/38a78d799aac4d4ab4c9c25f2ad5d930/users/835091aac8724ca5990bd892d68de552/roles/39e5c666285a42fe8fc2c3afca6260d1 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 05:44:05.430484 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-359601fc-3e9a-4f68-9926-a002ea602b41 admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:44:05.440292 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-359601fc-3e9a-4f68-9926-a002ea602b41 admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:44:05.499510 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d22f2339-3ca1-49dd-9c52-105eac73d19f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:44:05.500185 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22f2339-3ca1-49dd-9c52-105eac73d19f admin admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:44:05.500473 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22f2339-3ca1-49dd-9c52-105eac73d19f admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:44:05.500748 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22f2339-3ca1-49dd-9c52-105eac73d19f admin admin] PATH_INFO: `/v3/roles` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:44:05.501510 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d22f2339-3ca1-49dd-9c52-105eac73d19f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:44:05.503095 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d22f2339-3ca1-49dd-9c52-105eac73d19f admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:44:05.515371 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1266/2528] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jul 25 05:44:05 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 05:44:05.520241 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ff7569-0f0b-44c8-94dc-60dbf424b6dc admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:44:05.530676 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ff7569-0f0b-44c8-94dc-60dbf424b6dc admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:44:05.536013 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cec9f5a-8dab-49e4-9346-46d8116e1d4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:44:05.537002 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cec9f5a-8dab-49e4-9346-46d8116e1d4f admin admin] REQUEST_METHOD: `PUT` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:44:05.537286 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cec9f5a-8dab-49e4-9346-46d8116e1d4f admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:44:05.537563 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cec9f5a-8dab-49e4-9346-46d8116e1d4f admin admin] PATH_INFO: `/v3/projects/38a78d799aac4d4ab4c9c25f2ad5d930/users/835091aac8724ca5990bd892d68de552/roles/39e5c666285a42fe8fc2c3afca6260d1` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:44:05.539764 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cec9f5a-8dab-49e4-9346-46d8116e1d4f admin admin] RBAC: Authorizing `identity:create_grant(project_id=38a78d799aac4d4ab4c9c25f2ad5d930, user_id=835091aac8724ca5990bd892d68de552, role_id=39e5c666285a42fe8fc2c3afca6260d1)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:44:05.542528 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cec9f5a-8dab-49e4-9346-46d8116e1d4f admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:44:05.553296 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1263/2529] 10.222.0.33 () {60 vars in 1482 bytes} [Mon Jul 25 05:44:05 2022] PUT /identity/v3/projects/38a78d799aac4d4ab4c9c25f2ad5d930/users/835091aac8724ca5990bd892d68de552/roles/39e5c666285a42fe8fc2c3afca6260d1 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 05:44:05.563588 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a65df6-05be-4f57-845c-81af458d8a43 None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:44:05.563883 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a65df6-05be-4f57-845c-81af458d8a43 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:44:05.564167 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a65df6-05be-4f57-845c-81af458d8a43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:44:05.593817 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-77a65df6-05be-4f57-845c-81af458d8a43 None None] MFA Rules not processed for user `835091aac8724ca5990bd892d68de552`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:44:05.595180 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-77a65df6-05be-4f57-845c-81af458d8a43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:44:05.675145 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1267/2530] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 25 05:44:05 2022] POST /identity/v3/auth/tokens => generated 3289 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:44:05.686668 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cec9f5a-8dab-49e4-9346-46d8116e1d4f admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:44:05.688424 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-2cec9f5a-8dab-49e4-9346-46d8116e1d4f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:44:05.753535 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cec9f5a-8dab-49e4-9346-46d8116e1d4f admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:44:05.758749 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f389af94-2bfe-4a2b-9ad4-dd8abc52893f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db57b478c25643dbaf936f90f6e17d0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92afba266cce4d1bad4134c351e74be3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:44:05.760534 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f389af94-2bfe-4a2b-9ad4-dd8abc52893f service cinder] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:44:05.760890 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f389af94-2bfe-4a2b-9ad4-dd8abc52893f service cinder] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:44:05.761161 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f389af94-2bfe-4a2b-9ad4-dd8abc52893f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:44:05.773168 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f389af94-2bfe-4a2b-9ad4-dd8abc52893f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:44:05.774594 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f389af94-2bfe-4a2b-9ad4-dd8abc52893f service cinder] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:44:05.784721 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1264/2531] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jul 25 05:44:05 2022] GET /identity/v3/auth/tokens => generated 3289 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 05:44:05.891622 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a65df6-05be-4f57-845c-81af458d8a43 None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:44:05.902404 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a65df6-05be-4f57-845c-81af458d8a43 None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:44:05.906027 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c1f283d-7267-4a3b-8871-2637c399515c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db57b478c25643dbaf936f90f6e17d0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92afba266cce4d1bad4134c351e74be3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:44:05.906764 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1f283d-7267-4a3b-8871-2637c399515c service cinder] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:44:05.907178 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1f283d-7267-4a3b-8871-2637c399515c service cinder] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:44:05.907503 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1f283d-7267-4a3b-8871-2637c399515c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:44:05.967668 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c1f283d-7267-4a3b-8871-2637c399515c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:44:05.969163 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c1f283d-7267-4a3b-8871-2637c399515c service cinder] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:44:05.980651 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1268/2532] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jul 25 05:44:05 2022] GET /identity/v3/auth/tokens => generated 3224 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 05:44:15.780170 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=69,keystone:INSERT=5 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:44:15.976759 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=95 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:44:21.010370 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f389af94-2bfe-4a2b-9ad4-dd8abc52893f service cinder] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:44:21.071438 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f389af94-2bfe-4a2b-9ad4-dd8abc52893f service cinder] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:44:21.075895 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd37161-361a-4ee1-8c92-64a2410fd2be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:44:21.076760 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd37161-361a-4ee1-8c92-64a2410fd2be admin admin] REQUEST_METHOD: `DELETE` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:44:21.077067 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd37161-361a-4ee1-8c92-64a2410fd2be admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:44:21.077340 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd37161-361a-4ee1-8c92-64a2410fd2be admin admin] PATH_INFO: `/v3/users/6d0fcbedcff142a98c43f39e6a7ab0e4` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:44:21.078516 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbd37161-361a-4ee1-8c92-64a2410fd2be admin admin] RBAC: Authorizing `identity:delete_user(user_id=6d0fcbedcff142a98c43f39e6a7ab0e4)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:44:21.080414 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbd37161-361a-4ee1-8c92-64a2410fd2be admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:44:21.126492 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-dbd37161-361a-4ee1-8c92-64a2410fd2be admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6d0fcbedcff142a98c43f39e6a7ab0e4'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:44:21.127133 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-dbd37161-361a-4ee1-8c92-64a2410fd2be admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6d0fcbedcff142a98c43f39e6a7ab0e4'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:44:21.134044 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-dbd37161-361a-4ee1-8c92-64a2410fd2be admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6d0fcbedcff142a98c43f39e6a7ab0e4'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:44:21.145444 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-dbd37161-361a-4ee1-8c92-64a2410fd2be admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6d0fcbedcff142a98c43f39e6a7ab0e4'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:44:21.146113 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-dbd37161-361a-4ee1-8c92-64a2410fd2be admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6d0fcbedcff142a98c43f39e6a7ab0e4'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:44:21.158902 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1265/2533] 10.222.0.33 () {58 vars in 1223 bytes} [Mon Jul 25 05:44:21 2022] DELETE /identity/v3/users/6d0fcbedcff142a98c43f39e6a7ab0e4 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 05:44:21.164272 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5176229-be1c-40e3-86d0-144d90b50cce None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:44:21.164759 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5176229-be1c-40e3-86d0-144d90b50cce None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:44:21.164887 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5176229-be1c-40e3-86d0-144d90b50cce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:44:21.181517 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-b5176229-be1c-40e3-86d0-144d90b50cce None None] MFA Rules not processed for user `1eae540242d94c10b0ad33960a855f08`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:44:21.182763 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-b5176229-be1c-40e3-86d0-144d90b50cce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:44:21.240846 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1269/2534] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 25 05:44:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:44:21.250675 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd37161-361a-4ee1-8c92-64a2410fd2be admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:44:21.252119 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-dbd37161-361a-4ee1-8c92-64a2410fd2be admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:44:21.309709 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd37161-361a-4ee1-8c92-64a2410fd2be admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:44:21.313482 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa57d97-8edc-4838-9696-3a8b08314bf0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b2931200f99e43d79a50a9f6e02738e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92afba266cce4d1bad4134c351e74be3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:44:21.314125 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa57d97-8edc-4838-9696-3a8b08314bf0 service neutron] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:44:21.314405 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa57d97-8edc-4838-9696-3a8b08314bf0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:44:21.314678 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa57d97-8edc-4838-9696-3a8b08314bf0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:44:21.324760 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaa57d97-8edc-4838-9696-3a8b08314bf0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:44:21.326157 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaa57d97-8edc-4838-9696-3a8b08314bf0 service neutron] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:44:21.335506 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1266/2535] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jul 25 05:44:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 05:44:21.603113 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5176229-be1c-40e3-86d0-144d90b50cce None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:44:21.604559 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-b5176229-be1c-40e3-86d0-144d90b50cce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:44:21.616659 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5176229-be1c-40e3-86d0-144d90b50cce None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:44:21.621544 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c6b343c-224d-46ed-8c8d-dd97fd2a8417 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:44:21.622538 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6b343c-224d-46ed-8c8d-dd97fd2a8417 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:44:21.622836 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6b343c-224d-46ed-8c8d-dd97fd2a8417 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:44:21.623116 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6b343c-224d-46ed-8c8d-dd97fd2a8417 admin admin] PATH_INFO: `/v3/projects/f501377f72e843e7859efbb51e7a47f5` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:44:21.624357 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c6b343c-224d-46ed-8c8d-dd97fd2a8417 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f501377f72e843e7859efbb51e7a47f5)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:44:21.626331 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c6b343c-224d-46ed-8c8d-dd97fd2a8417 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:44:21.675833 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-3c6b343c-224d-46ed-8c8d-dd97fd2a8417 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f501377f72e843e7859efbb51e7a47f5', 'request_id': 'req-3c6b343c-224d-46ed-8c8d-dd97fd2a8417'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:44:21.682042 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-3c6b343c-224d-46ed-8c8d-dd97fd2a8417 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f501377f72e843e7859efbb51e7a47f5', 'request_id': 'req-3c6b343c-224d-46ed-8c8d-dd97fd2a8417'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:44:21.688193 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-3c6b343c-224d-46ed-8c8d-dd97fd2a8417 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f501377f72e843e7859efbb51e7a47f5', 'request_id': 'req-3c6b343c-224d-46ed-8c8d-dd97fd2a8417'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:44:21.701378 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-3c6b343c-224d-46ed-8c8d-dd97fd2a8417 admin admin] The token cache is being invalidate because project f501377f72e843e7859efbb51e7a47f5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80248) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 05:44:21.701662 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-3c6b343c-224d-46ed-8c8d-dd97fd2a8417 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:44:21.702269 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-3c6b343c-224d-46ed-8c8d-dd97fd2a8417 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:44:21.703823 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1270/2536] 10.222.0.33 () {58 vars in 1232 bytes} [Mon Jul 25 05:44:21 2022] DELETE /identity/v3/projects/f501377f72e843e7859efbb51e7a47f5 => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 05:44:21.707771 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa57d97-8edc-4838-9696-3a8b08314bf0 service neutron] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:44:21.718661 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa57d97-8edc-4838-9696-3a8b08314bf0 service neutron] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:44:21.739737 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:44:21.781739 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e216c8-717a-4815-8999-724b019646c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:44:21.782613 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e216c8-717a-4815-8999-724b019646c3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:44:21.782900 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e216c8-717a-4815-8999-724b019646c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:44:21.783173 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e216c8-717a-4815-8999-724b019646c3 admin admin] PATH_INFO: `/v3/users/835091aac8724ca5990bd892d68de552` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:44:21.784362 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97e216c8-717a-4815-8999-724b019646c3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=835091aac8724ca5990bd892d68de552)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:44:21.786357 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97e216c8-717a-4815-8999-724b019646c3 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:44:21.841072 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-97e216c8-717a-4815-8999-724b019646c3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '835091aac8724ca5990bd892d68de552'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:44:21.841730 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-97e216c8-717a-4815-8999-724b019646c3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '835091aac8724ca5990bd892d68de552'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:44:21.849047 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-97e216c8-717a-4815-8999-724b019646c3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '835091aac8724ca5990bd892d68de552'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:44:21.861725 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-97e216c8-717a-4815-8999-724b019646c3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '835091aac8724ca5990bd892d68de552'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:44:21.862424 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-97e216c8-717a-4815-8999-724b019646c3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '835091aac8724ca5990bd892d68de552'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:44:21.876164 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1267/2537] 10.222.0.33 () {58 vars in 1223 bytes} [Mon Jul 25 05:44:21 2022] DELETE /identity/v3/users/835091aac8724ca5990bd892d68de552 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 05:44:22.202489 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c6b343c-224d-46ed-8c8d-dd97fd2a8417 admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:44:22.204073 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-3c6b343c-224d-46ed-8c8d-dd97fd2a8417 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:44:22.217844 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c6b343c-224d-46ed-8c8d-dd97fd2a8417 admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:44:22.278800 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e68d11-a201-4717-aedd-c223faae6fb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:44:22.279658 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e68d11-a201-4717-aedd-c223faae6fb4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:44:22.279940 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e68d11-a201-4717-aedd-c223faae6fb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:44:22.280216 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e68d11-a201-4717-aedd-c223faae6fb4 admin admin] PATH_INFO: `/v3/projects/38a78d799aac4d4ab4c9c25f2ad5d930` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:44:22.281415 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92e68d11-a201-4717-aedd-c223faae6fb4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=38a78d799aac4d4ab4c9c25f2ad5d930)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:44:22.283307 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92e68d11-a201-4717-aedd-c223faae6fb4 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:44:22.313249 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:44:22.330649 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-92e68d11-a201-4717-aedd-c223faae6fb4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '38a78d799aac4d4ab4c9c25f2ad5d930', 'request_id': 'req-92e68d11-a201-4717-aedd-c223faae6fb4'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:44:22.336412 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-92e68d11-a201-4717-aedd-c223faae6fb4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '38a78d799aac4d4ab4c9c25f2ad5d930', 'request_id': 'req-92e68d11-a201-4717-aedd-c223faae6fb4'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:44:22.342654 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-92e68d11-a201-4717-aedd-c223faae6fb4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '38a78d799aac4d4ab4c9c25f2ad5d930', 'request_id': 'req-92e68d11-a201-4717-aedd-c223faae6fb4'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:44:22.355590 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-92e68d11-a201-4717-aedd-c223faae6fb4 admin admin] The token cache is being invalidate because project 38a78d799aac4d4ab4c9c25f2ad5d930 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80248) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 05:44:22.355878 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-92e68d11-a201-4717-aedd-c223faae6fb4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:44:22.356499 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-92e68d11-a201-4717-aedd-c223faae6fb4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:44:22.358098 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1271/2538] 10.222.0.33 () {58 vars in 1232 bytes} [Mon Jul 25 05:44:22 2022] DELETE /identity/v3/projects/38a78d799aac4d4ab4c9c25f2ad5d930 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 05:44:22.363183 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe399fd-a739-430e-9dd1-32ecf42dcdef None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:44:22.363654 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe399fd-a739-430e-9dd1-32ecf42dcdef None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:44:22.363774 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe399fd-a739-430e-9dd1-32ecf42dcdef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:44:22.382417 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-0fe399fd-a739-430e-9dd1-32ecf42dcdef None None] MFA Rules not processed for user `1eae540242d94c10b0ad33960a855f08`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:44:22.383631 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-0fe399fd-a739-430e-9dd1-32ecf42dcdef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:44:22.444298 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1268/2539] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 25 05:44:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:44:22.452717 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023a8a5c-3634-46fc-ac07-c7e1e1fee986 None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:44:22.453039 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023a8a5c-3634-46fc-ac07-c7e1e1fee986 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:44:22.453327 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023a8a5c-3634-46fc-ac07-c7e1e1fee986 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:44:22.475635 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-023a8a5c-3634-46fc-ac07-c7e1e1fee986 None None] MFA Rules not processed for user `1eae540242d94c10b0ad33960a855f08`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:44:22.477188 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-023a8a5c-3634-46fc-ac07-c7e1e1fee986 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:44:22.485136 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1272/2540] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 25 05:44:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:44:22.489543 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe399fd-a739-430e-9dd1-32ecf42dcdef None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:44:22.499870 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe399fd-a739-430e-9dd1-32ecf42dcdef None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:44:22.504433 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce039928-6325-4c69-88b1-99f3bc502683 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:44:22.505131 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce039928-6325-4c69-88b1-99f3bc502683 admin admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:44:22.505422 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce039928-6325-4c69-88b1-99f3bc502683 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:44:22.505705 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce039928-6325-4c69-88b1-99f3bc502683 admin admin] PATH_INFO: `/v3/domains` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:44:22.506519 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce039928-6325-4c69-88b1-99f3bc502683 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:44:22.506831 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce039928-6325-4c69-88b1-99f3bc502683 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:44:22.508733 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce039928-6325-4c69-88b1-99f3bc502683 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:44:22.520319 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1269/2541] 10.222.0.33 () {58 vars in 1152 bytes} [Mon Jul 25 05:44:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:44:22.525181 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-023a8a5c-3634-46fc-ac07-c7e1e1fee986 None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:44:22.534420 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-023a8a5c-3634-46fc-ac07-c7e1e1fee986 None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:44:22.539604 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b4d59d-1d33-4931-8640-e15c7584e8b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:44:22.540248 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b4d59d-1d33-4931-8640-e15c7584e8b8 admin admin] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:44:22.540529 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b4d59d-1d33-4931-8640-e15c7584e8b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:44:22.540830 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b4d59d-1d33-4931-8640-e15c7584e8b8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:44:22.541535 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0b4d59d-1d33-4931-8640-e15c7584e8b8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:44:22.543447 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0b4d59d-1d33-4931-8640-e15c7584e8b8 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:44:22.567922 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1273/2542] 10.222.0.33 () {60 vars in 1152 bytes} [Mon Jul 25 05:44:22 2022] POST /identity/v3/projects => generated 379 bytes in 43 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 05:44:22.572329 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce039928-6325-4c69-88b1-99f3bc502683 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:44:22.582400 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce039928-6325-4c69-88b1-99f3bc502683 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:44:22.610165 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:44:22.646605 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-964b17ac-f0be-4829-bf7b-9ab9f6d6673c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:44:22.646893 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964b17ac-f0be-4829-bf7b-9ab9f6d6673c admin admin] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:44:22.647174 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964b17ac-f0be-4829-bf7b-9ab9f6d6673c admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:44:22.647455 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964b17ac-f0be-4829-bf7b-9ab9f6d6673c admin admin] PATH_INFO: `/v3/users` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:44:22.648171 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-964b17ac-f0be-4829-bf7b-9ab9f6d6673c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:44:22.650127 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-964b17ac-f0be-4829-bf7b-9ab9f6d6673c admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:44:22.671292 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1270/2543] 10.222.0.33 () {60 vars in 1143 bytes} [Mon Jul 25 05:44:22 2022] POST /identity/v3/users => generated 290 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 05:44:22.675775 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b4d59d-1d33-4931-8640-e15c7584e8b8 admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:44:22.686964 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b4d59d-1d33-4931-8640-e15c7584e8b8 admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:44:22.691716 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c19df5bb-e65a-45db-b362-e03bf08f1043 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:44:22.692457 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19df5bb-e65a-45db-b362-e03bf08f1043 admin admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:44:22.692873 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19df5bb-e65a-45db-b362-e03bf08f1043 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:44:22.693219 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19df5bb-e65a-45db-b362-e03bf08f1043 admin admin] PATH_INFO: `/v3/roles` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:44:22.694645 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c19df5bb-e65a-45db-b362-e03bf08f1043 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:44:22.696317 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c19df5bb-e65a-45db-b362-e03bf08f1043 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:44:22.707561 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1274/2544] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jul 25 05:44:22 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 05:44:22.711743 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-964b17ac-f0be-4829-bf7b-9ab9f6d6673c admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:44:22.722821 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-964b17ac-f0be-4829-bf7b-9ab9f6d6673c admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:44:22.728157 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe6cf8f0-084a-4eaf-a1e0-3872d030e968 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:44:22.728849 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6cf8f0-084a-4eaf-a1e0-3872d030e968 admin admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:44:22.729134 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6cf8f0-084a-4eaf-a1e0-3872d030e968 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:44:22.729408 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6cf8f0-084a-4eaf-a1e0-3872d030e968 admin admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:44:22.730303 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe6cf8f0-084a-4eaf-a1e0-3872d030e968 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:44:22.731934 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe6cf8f0-084a-4eaf-a1e0-3872d030e968 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:44:22.743857 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1271/2545] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jul 25 05:44:22 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 05:44:22.748067 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c19df5bb-e65a-45db-b362-e03bf08f1043 admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:44:22.759048 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c19df5bb-e65a-45db-b362-e03bf08f1043 admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:44:22.764112 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c70153-7fa2-4f9f-8a42-bba131eaa6d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:44:22.765228 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c70153-7fa2-4f9f-8a42-bba131eaa6d6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:44:22.765514 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c70153-7fa2-4f9f-8a42-bba131eaa6d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:44:22.765799 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c70153-7fa2-4f9f-8a42-bba131eaa6d6 admin admin] PATH_INFO: `/v3/projects/ba35bce2dd8b43a08b5c02deed9a0b48/users/86a68004f7e14971a96cf6a6d6b1910a/roles/5074e74e35f54d4b9bb415c34999a82a` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:44:22.782281 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34c70153-7fa2-4f9f-8a42-bba131eaa6d6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ba35bce2dd8b43a08b5c02deed9a0b48, user_id=86a68004f7e14971a96cf6a6d6b1910a, role_id=5074e74e35f54d4b9bb415c34999a82a)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:44:22.784948 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34c70153-7fa2-4f9f-8a42-bba131eaa6d6 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:44:22.805046 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1275/2546] 10.222.0.33 () {60 vars in 1482 bytes} [Mon Jul 25 05:44:22 2022] PUT /identity/v3/projects/ba35bce2dd8b43a08b5c02deed9a0b48/users/86a68004f7e14971a96cf6a6d6b1910a/roles/5074e74e35f54d4b9bb415c34999a82a => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 05:44:22.816376 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679352e7-b648-4bbf-ab9c-473722eeef24 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:44:22.816669 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679352e7-b648-4bbf-ab9c-473722eeef24 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:44:22.817057 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679352e7-b648-4bbf-ab9c-473722eeef24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:44:22.844147 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-679352e7-b648-4bbf-ab9c-473722eeef24 None None] MFA Rules not processed for user `86a68004f7e14971a96cf6a6d6b1910a`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:44:22.845389 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-679352e7-b648-4bbf-ab9c-473722eeef24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:44:22.914967 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1272/2547] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 25 05:44:22 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:44:22.919324 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c70153-7fa2-4f9f-8a42-bba131eaa6d6 admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:44:22.994529 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c70153-7fa2-4f9f-8a42-bba131eaa6d6 admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:44:22.999485 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485906e7-0d1e-4259-9d0c-c3e0aca355e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:44:23.000360 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485906e7-0d1e-4259-9d0c-c3e0aca355e6 admin admin] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:44:23.000646 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485906e7-0d1e-4259-9d0c-c3e0aca355e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:44:23.001037 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485906e7-0d1e-4259-9d0c-c3e0aca355e6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:44:23.001823 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-485906e7-0d1e-4259-9d0c-c3e0aca355e6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:44:23.004169 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-485906e7-0d1e-4259-9d0c-c3e0aca355e6 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:44:23.039795 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1276/2548] 10.222.0.33 () {60 vars in 1152 bytes} [Mon Jul 25 05:44:22 2022] POST /identity/v3/projects => generated 377 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 05:44:23.041031 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-679352e7-b648-4bbf-ab9c-473722eeef24 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:44:23.055329 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-679352e7-b648-4bbf-ab9c-473722eeef24 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:44:23.113298 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e7fd89-e29e-4203-868b-fc0e5a482bf0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:44:23.113970 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e7fd89-e29e-4203-868b-fc0e5a482bf0 admin admin] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:44:23.114261 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e7fd89-e29e-4203-868b-fc0e5a482bf0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:44:23.114538 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e7fd89-e29e-4203-868b-fc0e5a482bf0 admin admin] PATH_INFO: `/v3/users` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:44:23.115229 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83e7fd89-e29e-4203-868b-fc0e5a482bf0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:44:23.117896 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83e7fd89-e29e-4203-868b-fc0e5a482bf0 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:44:23.139178 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1273/2549] 10.222.0.33 () {60 vars in 1143 bytes} [Mon Jul 25 05:44:23 2022] POST /identity/v3/users => generated 289 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 05:44:23.143463 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485906e7-0d1e-4259-9d0c-c3e0aca355e6 admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:44:23.155502 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485906e7-0d1e-4259-9d0c-c3e0aca355e6 admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:44:23.161719 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-149642bb-0f66-4f2a-bc1a-7bb593264243 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:44:23.163559 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149642bb-0f66-4f2a-bc1a-7bb593264243 admin admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:44:23.163848 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149642bb-0f66-4f2a-bc1a-7bb593264243 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:44:23.164128 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149642bb-0f66-4f2a-bc1a-7bb593264243 admin admin] PATH_INFO: `/v3/roles` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:44:23.164902 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-149642bb-0f66-4f2a-bc1a-7bb593264243 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:44:23.166539 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-149642bb-0f66-4f2a-bc1a-7bb593264243 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:44:23.200734 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1277/2550] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jul 25 05:44:23 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 05:44:23.205913 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e7fd89-e29e-4203-868b-fc0e5a482bf0 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:44:23.221389 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e7fd89-e29e-4203-868b-fc0e5a482bf0 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:44:23.228868 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16754444-8eb0-403f-97a8-e31a02294e77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:44:23.229650 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16754444-8eb0-403f-97a8-e31a02294e77 admin admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:44:23.230000 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16754444-8eb0-403f-97a8-e31a02294e77 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:44:23.230344 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16754444-8eb0-403f-97a8-e31a02294e77 admin admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:44:23.231157 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16754444-8eb0-403f-97a8-e31a02294e77 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:44:23.232843 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16754444-8eb0-403f-97a8-e31a02294e77 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:44:23.244135 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1274/2551] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jul 25 05:44:23 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 05:44:23.248072 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-149642bb-0f66-4f2a-bc1a-7bb593264243 admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:44:23.260176 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-149642bb-0f66-4f2a-bc1a-7bb593264243 admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:44:23.264977 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e684d92-6d1a-41fe-8e04-2c8aa5ada921 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:44:23.265908 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e684d92-6d1a-41fe-8e04-2c8aa5ada921 admin admin] REQUEST_METHOD: `PUT` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:44:23.266198 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e684d92-6d1a-41fe-8e04-2c8aa5ada921 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:44:23.266476 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e684d92-6d1a-41fe-8e04-2c8aa5ada921 admin admin] PATH_INFO: `/v3/projects/a8b49b2b40c04cf0bcb02b004a1516bd/users/723b6bc8d8ee43788dad3ffc2bf9c3b5/roles/5074e74e35f54d4b9bb415c34999a82a` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:44:23.284104 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e684d92-6d1a-41fe-8e04-2c8aa5ada921 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a8b49b2b40c04cf0bcb02b004a1516bd, user_id=723b6bc8d8ee43788dad3ffc2bf9c3b5, role_id=5074e74e35f54d4b9bb415c34999a82a)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:44:23.286773 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e684d92-6d1a-41fe-8e04-2c8aa5ada921 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:44:23.298456 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:DELETE=6,keystone:SELECT=89,keystone:INSERT=5 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:44:23.310066 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1278/2552] 10.222.0.33 () {60 vars in 1482 bytes} [Mon Jul 25 05:44:23 2022] PUT /identity/v3/projects/a8b49b2b40c04cf0bcb02b004a1516bd/users/723b6bc8d8ee43788dad3ffc2bf9c3b5/roles/5074e74e35f54d4b9bb415c34999a82a => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 05:44:23.316000 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d4d8a0-dd59-4bfa-94a8-bf09067eea5b None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:44:23.316359 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d4d8a0-dd59-4bfa-94a8-bf09067eea5b None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:44:23.316711 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d4d8a0-dd59-4bfa-94a8-bf09067eea5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:44:23.344429 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-89d4d8a0-dd59-4bfa-94a8-bf09067eea5b None None] MFA Rules not processed for user `723b6bc8d8ee43788dad3ffc2bf9c3b5`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:44:23.345697 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-89d4d8a0-dd59-4bfa-94a8-bf09067eea5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:44:23.352195 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:44:23.416387 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1275/2553] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 25 05:44:23 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:44:23.420947 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e684d92-6d1a-41fe-8e04-2c8aa5ada921 admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:44:23.492892 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e684d92-6d1a-41fe-8e04-2c8aa5ada921 admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:44:23.497786 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-179a5831-057f-4e07-9eff-6d2721844811 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:44:23.498453 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179a5831-057f-4e07-9eff-6d2721844811 admin admin] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:44:23.498828 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179a5831-057f-4e07-9eff-6d2721844811 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:44:23.499000 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179a5831-057f-4e07-9eff-6d2721844811 admin admin] PATH_INFO: `/v3/projects` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:44:23.499716 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-179a5831-057f-4e07-9eff-6d2721844811 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:44:23.501632 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-179a5831-057f-4e07-9eff-6d2721844811 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:44:23.534173 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1279/2554] 10.222.0.33 () {60 vars in 1152 bytes} [Mon Jul 25 05:44:23 2022] POST /identity/v3/projects => generated 379 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 05:44:23.540329 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d4d8a0-dd59-4bfa-94a8-bf09067eea5b None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:44:23.555186 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d4d8a0-dd59-4bfa-94a8-bf09067eea5b None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:44:23.623306 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45cce667-b4b0-495e-8b94-ff108e5a2835 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:44:23.624816 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45cce667-b4b0-495e-8b94-ff108e5a2835 admin admin] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:44:23.625098 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45cce667-b4b0-495e-8b94-ff108e5a2835 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:44:23.625521 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45cce667-b4b0-495e-8b94-ff108e5a2835 admin admin] PATH_INFO: `/v3/users` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:44:23.626606 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45cce667-b4b0-495e-8b94-ff108e5a2835 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:44:23.628555 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45cce667-b4b0-495e-8b94-ff108e5a2835 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:44:23.659973 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1276/2555] 10.222.0.33 () {60 vars in 1143 bytes} [Mon Jul 25 05:44:23 2022] POST /identity/v3/users => generated 296 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 05:44:23.662172 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-179a5831-057f-4e07-9eff-6d2721844811 admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:44:23.676935 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-179a5831-057f-4e07-9eff-6d2721844811 admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:44:23.684026 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ebfeda3-9876-467c-aff7-6036118cb65f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:44:23.685136 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ebfeda3-9876-467c-aff7-6036118cb65f admin admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:44:23.685475 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ebfeda3-9876-467c-aff7-6036118cb65f admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:44:23.685900 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ebfeda3-9876-467c-aff7-6036118cb65f admin admin] PATH_INFO: `/v3/roles` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:44:23.687921 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ebfeda3-9876-467c-aff7-6036118cb65f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:44:23.690881 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ebfeda3-9876-467c-aff7-6036118cb65f admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:44:23.711578 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1280/2556] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jul 25 05:44:23 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 05:44:23.712861 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45cce667-b4b0-495e-8b94-ff108e5a2835 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:44:23.727483 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45cce667-b4b0-495e-8b94-ff108e5a2835 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:44:23.732920 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b29ba3c-33b2-4bc9-918f-0f08de1305fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:44:23.734520 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b29ba3c-33b2-4bc9-918f-0f08de1305fb admin admin] REQUEST_METHOD: `PUT` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:44:23.734804 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b29ba3c-33b2-4bc9-918f-0f08de1305fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:44:23.735083 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b29ba3c-33b2-4bc9-918f-0f08de1305fb admin admin] PATH_INFO: `/v3/projects/0d696daf085644289ba693a7be5b9213/users/96916608764c46d1bedef0f0f233f41e/roles/39e5c666285a42fe8fc2c3afca6260d1` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:44:23.755565 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b29ba3c-33b2-4bc9-918f-0f08de1305fb admin admin] RBAC: Authorizing `identity:create_grant(project_id=0d696daf085644289ba693a7be5b9213, user_id=96916608764c46d1bedef0f0f233f41e, role_id=39e5c666285a42fe8fc2c3afca6260d1)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:44:23.760088 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b29ba3c-33b2-4bc9-918f-0f08de1305fb admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:44:23.775047 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1277/2557] 10.222.0.33 () {60 vars in 1482 bytes} [Mon Jul 25 05:44:23 2022] PUT /identity/v3/projects/0d696daf085644289ba693a7be5b9213/users/96916608764c46d1bedef0f0f233f41e/roles/39e5c666285a42fe8fc2c3afca6260d1 => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 05:44:23.779057 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ebfeda3-9876-467c-aff7-6036118cb65f admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:44:23.789438 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ebfeda3-9876-467c-aff7-6036118cb65f admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:44:23.854767 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9043a71-7927-46ba-a702-c8702a7f33db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:44:23.855436 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9043a71-7927-46ba-a702-c8702a7f33db admin admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:44:23.855716 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9043a71-7927-46ba-a702-c8702a7f33db admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:44:23.855993 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9043a71-7927-46ba-a702-c8702a7f33db admin admin] PATH_INFO: `/v3/roles` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:44:23.856719 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9043a71-7927-46ba-a702-c8702a7f33db admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:44:23.858351 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9043a71-7927-46ba-a702-c8702a7f33db admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:44:23.872589 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1281/2558] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jul 25 05:44:23 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 25 05:44:23.873826 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b29ba3c-33b2-4bc9-918f-0f08de1305fb admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:44:23.886710 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b29ba3c-33b2-4bc9-918f-0f08de1305fb admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:44:23.891431 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b748f7d-e81b-4a10-bf66-f0f0cc3dd1f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:44:23.892316 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b748f7d-e81b-4a10-bf66-f0f0cc3dd1f6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:44:23.892596 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b748f7d-e81b-4a10-bf66-f0f0cc3dd1f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:44:23.892906 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b748f7d-e81b-4a10-bf66-f0f0cc3dd1f6 admin admin] PATH_INFO: `/v3/projects/0d696daf085644289ba693a7be5b9213/users/96916608764c46d1bedef0f0f233f41e/roles/39e5c666285a42fe8fc2c3afca6260d1` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:44:23.894879 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b748f7d-e81b-4a10-bf66-f0f0cc3dd1f6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0d696daf085644289ba693a7be5b9213, user_id=96916608764c46d1bedef0f0f233f41e, role_id=39e5c666285a42fe8fc2c3afca6260d1)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:44:23.898366 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b748f7d-e81b-4a10-bf66-f0f0cc3dd1f6 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:44:23.909140 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1278/2559] 10.222.0.33 () {60 vars in 1482 bytes} [Mon Jul 25 05:44:23 2022] PUT /identity/v3/projects/0d696daf085644289ba693a7be5b9213/users/96916608764c46d1bedef0f0f233f41e/roles/39e5c666285a42fe8fc2c3afca6260d1 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 05:44:23.921290 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a69900-a0ad-4c7a-a08c-8b933619cc3d None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:44:23.921527 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a69900-a0ad-4c7a-a08c-8b933619cc3d None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:44:23.921808 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a69900-a0ad-4c7a-a08c-8b933619cc3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:44:23.949587 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-73a69900-a0ad-4c7a-a08c-8b933619cc3d None None] MFA Rules not processed for user `96916608764c46d1bedef0f0f233f41e`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:44:23.950796 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-73a69900-a0ad-4c7a-a08c-8b933619cc3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:44:24.010436 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:44:24.021165 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1282/2560] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 25 05:44:23 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:44:24.036226 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b748f7d-e81b-4a10-bf66-f0f0cc3dd1f6 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:44:24.038645 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-9b748f7d-e81b-4a10-bf66-f0f0cc3dd1f6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:44:24.112303 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b748f7d-e81b-4a10-bf66-f0f0cc3dd1f6 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:44:24.117602 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd8316e-b3f8-4edc-9671-facd3a69b2e9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db57b478c25643dbaf936f90f6e17d0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92afba266cce4d1bad4134c351e74be3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:44:24.118554 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd8316e-b3f8-4edc-9671-facd3a69b2e9 service cinder] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:44:24.118982 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd8316e-b3f8-4edc-9671-facd3a69b2e9 service cinder] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:44:24.119561 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd8316e-b3f8-4edc-9671-facd3a69b2e9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:44:24.137495 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dd8316e-b3f8-4edc-9671-facd3a69b2e9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:44:24.138966 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dd8316e-b3f8-4edc-9671-facd3a69b2e9 service cinder] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:44:24.147735 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:44:24.152813 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1279/2561] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jul 25 05:44:24 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 05:44:24.230334 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a69900-a0ad-4c7a-a08c-8b933619cc3d None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:44:24.232377 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-73a69900-a0ad-4c7a-a08c-8b933619cc3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:44:24.246637 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a69900-a0ad-4c7a-a08c-8b933619cc3d None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:44:24.252034 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c312cf64-6cc6-4caa-96e8-2b8eb079fd20 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db57b478c25643dbaf936f90f6e17d0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92afba266cce4d1bad4134c351e74be3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:44:24.252612 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c312cf64-6cc6-4caa-96e8-2b8eb079fd20 service cinder] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:44:24.252860 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c312cf64-6cc6-4caa-96e8-2b8eb079fd20 service cinder] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:44:24.253138 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c312cf64-6cc6-4caa-96e8-2b8eb079fd20 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:44:24.318490 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c312cf64-6cc6-4caa-96e8-2b8eb079fd20 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:44:24.320569 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c312cf64-6cc6-4caa-96e8-2b8eb079fd20 service cinder] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:44:24.330197 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1283/2562] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jul 25 05:44:24 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 05:44:26.522974 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41bcf9f2-8c65-4f0a-a359-b455a74f9a2f None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:44:26.523541 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41bcf9f2-8c65-4f0a-a359-b455a74f9a2f None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:44:26.523541 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41bcf9f2-8c65-4f0a-a359-b455a74f9a2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:44:26.541656 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-41bcf9f2-8c65-4f0a-a359-b455a74f9a2f None None] MFA Rules not processed for user `3352531be1b54f76b864383ccc2e5ec2`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:44:26.542921 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-41bcf9f2-8c65-4f0a-a359-b455a74f9a2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:44:26.601783 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1280/2563] 10.222.0.33 () {58 vars in 1009 bytes} [Mon Jul 25 05:44:26 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:44:26.648097 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c312cf64-6cc6-4caa-96e8-2b8eb079fd20 service cinder] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:44:26.649602 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-c312cf64-6cc6-4caa-96e8-2b8eb079fd20 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:44:26.728198 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c312cf64-6cc6-4caa-96e8-2b8eb079fd20 service cinder] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:44:26.733914 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-351d1387-8fb8-4be2-94d2-d68e89ecc4af service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb135a8d5d7444cfbfa336115ba2ba44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92afba266cce4d1bad4134c351e74be3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:44:26.734658 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351d1387-8fb8-4be2-94d2-d68e89ecc4af service swift] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:44:26.734947 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351d1387-8fb8-4be2-94d2-d68e89ecc4af service swift] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:44:26.735223 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351d1387-8fb8-4be2-94d2-d68e89ecc4af service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:44:26.745181 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-351d1387-8fb8-4be2-94d2-d68e89ecc4af service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:44:26.746841 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-351d1387-8fb8-4be2-94d2-d68e89ecc4af service swift] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:44:26.756355 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1284/2564] 10.222.0.33 () {60 vars in 1384 bytes} [Mon Jul 25 05:44:26 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 25 05:44:27.343428 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41bcf9f2-8c65-4f0a-a359-b455a74f9a2f None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:44:27.478043 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41bcf9f2-8c65-4f0a-a359-b455a74f9a2f None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:44:27.486598 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e9694e-b2b5-43dd-9925-6fa9c1a3e109 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:44:27.489781 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e9694e-b2b5-43dd-9925-6fa9c1a3e109 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:44:27.490072 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e9694e-b2b5-43dd-9925-6fa9c1a3e109 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:44:27.490427 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e9694e-b2b5-43dd-9925-6fa9c1a3e109 admin admin] PATH_INFO: `/v3/users/86a68004f7e14971a96cf6a6d6b1910a` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:44:27.492357 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4e9694e-b2b5-43dd-9925-6fa9c1a3e109 admin admin] RBAC: Authorizing `identity:delete_user(user_id=86a68004f7e14971a96cf6a6d6b1910a)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:44:27.494977 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4e9694e-b2b5-43dd-9925-6fa9c1a3e109 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:44:27.561996 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-c4e9694e-b2b5-43dd-9925-6fa9c1a3e109 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '86a68004f7e14971a96cf6a6d6b1910a'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:44:27.562802 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-c4e9694e-b2b5-43dd-9925-6fa9c1a3e109 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '86a68004f7e14971a96cf6a6d6b1910a'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:44:27.571961 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-c4e9694e-b2b5-43dd-9925-6fa9c1a3e109 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '86a68004f7e14971a96cf6a6d6b1910a'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:44:27.589900 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-c4e9694e-b2b5-43dd-9925-6fa9c1a3e109 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '86a68004f7e14971a96cf6a6d6b1910a'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:44:27.591019 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-c4e9694e-b2b5-43dd-9925-6fa9c1a3e109 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '86a68004f7e14971a96cf6a6d6b1910a'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:44:27.612207 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1281/2565] 10.222.0.33 () {58 vars in 1223 bytes} [Mon Jul 25 05:44:27 2022] DELETE /identity/v3/users/86a68004f7e14971a96cf6a6d6b1910a => generated 0 bytes in 269 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 05:44:27.617274 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b3888b-fe5b-4dff-9705-da4b608f87a5 None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:44:27.618496 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b3888b-fe5b-4dff-9705-da4b608f87a5 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:44:27.618496 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b3888b-fe5b-4dff-9705-da4b608f87a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:44:27.641288 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-39b3888b-fe5b-4dff-9705-da4b608f87a5 None None] MFA Rules not processed for user `1eae540242d94c10b0ad33960a855f08`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:44:27.642633 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-39b3888b-fe5b-4dff-9705-da4b608f87a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:44:27.721752 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1285/2566] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 25 05:44:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 05:44:27.734068 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e9694e-b2b5-43dd-9925-6fa9c1a3e109 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:44:27.735660 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-c4e9694e-b2b5-43dd-9925-6fa9c1a3e109 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:44:27.782644 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:44:27.828819 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e9694e-b2b5-43dd-9925-6fa9c1a3e109 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:44:27.831939 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4bbb2b8-b3fe-4d59-a07f-9f3fb327f4a2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b2931200f99e43d79a50a9f6e02738e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92afba266cce4d1bad4134c351e74be3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:44:27.832603 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4bbb2b8-b3fe-4d59-a07f-9f3fb327f4a2 service neutron] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:44:27.832977 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4bbb2b8-b3fe-4d59-a07f-9f3fb327f4a2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:44:27.833265 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4bbb2b8-b3fe-4d59-a07f-9f3fb327f4a2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:44:27.844277 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4bbb2b8-b3fe-4d59-a07f-9f3fb327f4a2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:44:27.845752 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4bbb2b8-b3fe-4d59-a07f-9f3fb327f4a2 service neutron] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:44:27.862342 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1282/2567] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jul 25 05:44:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 05:44:28.194551 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b3888b-fe5b-4dff-9705-da4b608f87a5 None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:44:28.196231 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-39b3888b-fe5b-4dff-9705-da4b608f87a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:44:28.219662 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b3888b-fe5b-4dff-9705-da4b608f87a5 None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:44:28.226882 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efab3f4f-81d5-4296-aa98-4389e875cb8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:44:28.228621 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efab3f4f-81d5-4296-aa98-4389e875cb8e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:44:28.229042 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efab3f4f-81d5-4296-aa98-4389e875cb8e admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:44:28.229643 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efab3f4f-81d5-4296-aa98-4389e875cb8e admin admin] PATH_INFO: `/v3/projects/ba35bce2dd8b43a08b5c02deed9a0b48` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:44:28.231022 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efab3f4f-81d5-4296-aa98-4389e875cb8e admin admin] RBAC: Authorizing `identity:delete_project(project_id=ba35bce2dd8b43a08b5c02deed9a0b48)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:44:28.233555 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efab3f4f-81d5-4296-aa98-4389e875cb8e admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:44:28.273066 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:44:28.330771 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-efab3f4f-81d5-4296-aa98-4389e875cb8e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ba35bce2dd8b43a08b5c02deed9a0b48', 'request_id': 'req-efab3f4f-81d5-4296-aa98-4389e875cb8e'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:44:28.341735 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-efab3f4f-81d5-4296-aa98-4389e875cb8e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ba35bce2dd8b43a08b5c02deed9a0b48', 'request_id': 'req-efab3f4f-81d5-4296-aa98-4389e875cb8e'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:44:28.348944 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-efab3f4f-81d5-4296-aa98-4389e875cb8e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ba35bce2dd8b43a08b5c02deed9a0b48', 'request_id': 'req-efab3f4f-81d5-4296-aa98-4389e875cb8e'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:44:28.382227 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-efab3f4f-81d5-4296-aa98-4389e875cb8e admin admin] The token cache is being invalidate because project ba35bce2dd8b43a08b5c02deed9a0b48 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80248) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 05:44:28.382633 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-efab3f4f-81d5-4296-aa98-4389e875cb8e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:44:28.383818 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-efab3f4f-81d5-4296-aa98-4389e875cb8e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:44:28.388815 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1286/2568] 10.222.0.33 () {58 vars in 1232 bytes} [Mon Jul 25 05:44:28 2022] DELETE /identity/v3/projects/ba35bce2dd8b43a08b5c02deed9a0b48 => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 05:44:28.389974 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4bbb2b8-b3fe-4d59-a07f-9f3fb327f4a2 service neutron] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:44:28.404109 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4bbb2b8-b3fe-4d59-a07f-9f3fb327f4a2 service neutron] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:44:28.519154 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b11c6f-07dd-4be6-a729-60d27ea106aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:44:28.519886 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b11c6f-07dd-4be6-a729-60d27ea106aa admin admin] REQUEST_METHOD: `DELETE` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:44:28.520304 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b11c6f-07dd-4be6-a729-60d27ea106aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:44:28.520593 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b11c6f-07dd-4be6-a729-60d27ea106aa admin admin] PATH_INFO: `/v3/users/723b6bc8d8ee43788dad3ffc2bf9c3b5` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:44:28.525645 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90b11c6f-07dd-4be6-a729-60d27ea106aa admin admin] RBAC: Authorizing `identity:delete_user(user_id=723b6bc8d8ee43788dad3ffc2bf9c3b5)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:44:28.528411 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90b11c6f-07dd-4be6-a729-60d27ea106aa admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:44:28.598240 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-90b11c6f-07dd-4be6-a729-60d27ea106aa admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '723b6bc8d8ee43788dad3ffc2bf9c3b5'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:44:28.599760 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-90b11c6f-07dd-4be6-a729-60d27ea106aa admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '723b6bc8d8ee43788dad3ffc2bf9c3b5'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:44:28.614253 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-90b11c6f-07dd-4be6-a729-60d27ea106aa admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '723b6bc8d8ee43788dad3ffc2bf9c3b5'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:44:28.633772 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-90b11c6f-07dd-4be6-a729-60d27ea106aa admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '723b6bc8d8ee43788dad3ffc2bf9c3b5'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:44:28.636379 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-90b11c6f-07dd-4be6-a729-60d27ea106aa admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '723b6bc8d8ee43788dad3ffc2bf9c3b5'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:44:28.661001 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1283/2569] 10.222.0.33 () {58 vars in 1223 bytes} [Mon Jul 25 05:44:28 2022] DELETE /identity/v3/users/723b6bc8d8ee43788dad3ffc2bf9c3b5 => generated 0 bytes in 268 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 05:44:29.003280 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efab3f4f-81d5-4296-aa98-4389e875cb8e admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:44:29.005379 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-efab3f4f-81d5-4296-aa98-4389e875cb8e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:44:29.044136 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efab3f4f-81d5-4296-aa98-4389e875cb8e admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:44:29.122930 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c50f507c-4ee6-4224-add3-84e4d4d4614c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:44:29.123804 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50f507c-4ee6-4224-add3-84e4d4d4614c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:44:29.124085 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50f507c-4ee6-4224-add3-84e4d4d4614c admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:44:29.124363 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50f507c-4ee6-4224-add3-84e4d4d4614c admin admin] PATH_INFO: `/v3/projects/a8b49b2b40c04cf0bcb02b004a1516bd` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:44:29.125581 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c50f507c-4ee6-4224-add3-84e4d4d4614c admin admin] RBAC: Authorizing `identity:delete_project(project_id=a8b49b2b40c04cf0bcb02b004a1516bd)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:44:29.127673 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c50f507c-4ee6-4224-add3-84e4d4d4614c admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:44:29.183103 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-c50f507c-4ee6-4224-add3-84e4d4d4614c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a8b49b2b40c04cf0bcb02b004a1516bd', 'request_id': 'req-c50f507c-4ee6-4224-add3-84e4d4d4614c'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:44:29.191017 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-c50f507c-4ee6-4224-add3-84e4d4d4614c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a8b49b2b40c04cf0bcb02b004a1516bd', 'request_id': 'req-c50f507c-4ee6-4224-add3-84e4d4d4614c'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:44:29.199468 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-c50f507c-4ee6-4224-add3-84e4d4d4614c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a8b49b2b40c04cf0bcb02b004a1516bd', 'request_id': 'req-c50f507c-4ee6-4224-add3-84e4d4d4614c'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:44:29.214189 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-c50f507c-4ee6-4224-add3-84e4d4d4614c admin admin] The token cache is being invalidate because project a8b49b2b40c04cf0bcb02b004a1516bd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80248) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 05:44:29.214653 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-c50f507c-4ee6-4224-add3-84e4d4d4614c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:44:29.215438 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-c50f507c-4ee6-4224-add3-84e4d4d4614c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:44:29.220144 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1287/2570] 10.222.0.33 () {58 vars in 1232 bytes} [Mon Jul 25 05:44:29 2022] DELETE /identity/v3/projects/a8b49b2b40c04cf0bcb02b004a1516bd => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 05:44:29.233886 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b11c6f-07dd-4be6-a729-60d27ea106aa admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:44:29.235476 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-90b11c6f-07dd-4be6-a729-60d27ea106aa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:44:29.251217 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b11c6f-07dd-4be6-a729-60d27ea106aa admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:44:29.313599 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:44:29.338807 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24200947-79d1-4b47-a54d-bc35c8761c03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:44:29.340077 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24200947-79d1-4b47-a54d-bc35c8761c03 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:44:29.340434 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24200947-79d1-4b47-a54d-bc35c8761c03 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:44:29.340805 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24200947-79d1-4b47-a54d-bc35c8761c03 admin admin] PATH_INFO: `/v3/users/96916608764c46d1bedef0f0f233f41e` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:44:29.342196 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24200947-79d1-4b47-a54d-bc35c8761c03 admin admin] RBAC: Authorizing `identity:delete_user(user_id=96916608764c46d1bedef0f0f233f41e)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:44:29.344211 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24200947-79d1-4b47-a54d-bc35c8761c03 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:44:29.545381 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-24200947-79d1-4b47-a54d-bc35c8761c03 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '96916608764c46d1bedef0f0f233f41e'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:44:29.546209 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-24200947-79d1-4b47-a54d-bc35c8761c03 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '96916608764c46d1bedef0f0f233f41e'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:44:29.556555 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-24200947-79d1-4b47-a54d-bc35c8761c03 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '96916608764c46d1bedef0f0f233f41e'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:44:29.582480 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-24200947-79d1-4b47-a54d-bc35c8761c03 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '96916608764c46d1bedef0f0f233f41e'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:44:29.583342 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-24200947-79d1-4b47-a54d-bc35c8761c03 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '96916608764c46d1bedef0f0f233f41e'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:44:29.604809 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1284/2571] 10.222.0.33 () {58 vars in 1223 bytes} [Mon Jul 25 05:44:29 2022] DELETE /identity/v3/users/96916608764c46d1bedef0f0f233f41e => generated 0 bytes in 369 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 05:44:29.903295 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c50f507c-4ee6-4224-add3-84e4d4d4614c admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:44:29.904853 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-c50f507c-4ee6-4224-add3-84e4d4d4614c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:44:29.968426 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:44:29.992769 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c50f507c-4ee6-4224-add3-84e4d4d4614c admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:44:30.041580 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24200947-79d1-4b47-a54d-bc35c8761c03 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:44:30.043084 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-24200947-79d1-4b47-a54d-bc35c8761c03 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:44:30.055974 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d28e45c3-fb7e-43d0-8548-2d0bd7cc98d3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b2931200f99e43d79a50a9f6e02738e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92afba266cce4d1bad4134c351e74be3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:44:30.056741 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28e45c3-fb7e-43d0-8548-2d0bd7cc98d3 service neutron] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:44:30.057438 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28e45c3-fb7e-43d0-8548-2d0bd7cc98d3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:44:30.057438 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28e45c3-fb7e-43d0-8548-2d0bd7cc98d3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:44:30.063659 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-d28e45c3-fb7e-43d0-8548-2d0bd7cc98d3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:44:30.144661 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d28e45c3-fb7e-43d0-8548-2d0bd7cc98d3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:44:30.146302 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d28e45c3-fb7e-43d0-8548-2d0bd7cc98d3 service neutron] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:44:30.156060 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24200947-79d1-4b47-a54d-bc35c8761c03 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:44:30.160757 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a560cc-adc8-4598-aff4-478a3b6fb589 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:44:30.161854 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a560cc-adc8-4598-aff4-478a3b6fb589 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:44:30.162140 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a560cc-adc8-4598-aff4-478a3b6fb589 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:44:30.162414 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a560cc-adc8-4598-aff4-478a3b6fb589 admin admin] PATH_INFO: `/v3/projects/0d696daf085644289ba693a7be5b9213` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:44:30.164232 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26a560cc-adc8-4598-aff4-478a3b6fb589 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0d696daf085644289ba693a7be5b9213)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:44:30.167442 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26a560cc-adc8-4598-aff4-478a3b6fb589 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:44:30.167599 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1288/2572] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jul 25 05:44:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 264 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 05:44:30.244488 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-26a560cc-adc8-4598-aff4-478a3b6fb589 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0d696daf085644289ba693a7be5b9213', 'request_id': 'req-26a560cc-adc8-4598-aff4-478a3b6fb589'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:44:30.254884 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-26a560cc-adc8-4598-aff4-478a3b6fb589 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0d696daf085644289ba693a7be5b9213', 'request_id': 'req-26a560cc-adc8-4598-aff4-478a3b6fb589'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:44:30.268570 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-26a560cc-adc8-4598-aff4-478a3b6fb589 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0d696daf085644289ba693a7be5b9213', 'request_id': 'req-26a560cc-adc8-4598-aff4-478a3b6fb589'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:44:30.273290 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:44:30.293764 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-26a560cc-adc8-4598-aff4-478a3b6fb589 admin admin] The token cache is being invalidate because project 0d696daf085644289ba693a7be5b9213 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80249) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 05:44:30.294309 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-26a560cc-adc8-4598-aff4-478a3b6fb589 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:44:30.295012 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-26a560cc-adc8-4598-aff4-478a3b6fb589 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:44:30.297197 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1285/2573] 10.222.0.33 () {58 vars in 1232 bytes} [Mon Jul 25 05:44:30 2022] DELETE /identity/v3/projects/0d696daf085644289ba693a7be5b9213 => generated 0 bytes in 256 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 05:44:30.302846 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d5b8d8-b2fb-4761-a34c-bd2164eae5b7 None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:44:30.305958 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d5b8d8-b2fb-4761-a34c-bd2164eae5b7 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:44:30.307539 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d5b8d8-b2fb-4761-a34c-bd2164eae5b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:44:30.332466 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-97d5b8d8-b2fb-4761-a34c-bd2164eae5b7 None None] MFA Rules not processed for user `1eae540242d94c10b0ad33960a855f08`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:44:30.333875 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-97d5b8d8-b2fb-4761-a34c-bd2164eae5b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:44:30.432048 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1289/2574] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 25 05:44:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 05:44:30.432665 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc28b34-3d16-4bbf-85e8-2853a39380d1 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:44:30.433008 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc28b34-3d16-4bbf-85e8-2853a39380d1 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:44:30.433291 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc28b34-3d16-4bbf-85e8-2853a39380d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:44:30.456260 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-0dc28b34-3d16-4bbf-85e8-2853a39380d1 None None] MFA Rules not processed for user `1eae540242d94c10b0ad33960a855f08`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:44:30.457673 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-0dc28b34-3d16-4bbf-85e8-2853a39380d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:44:30.469356 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1286/2575] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 25 05:44:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:44:30.473860 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d5b8d8-b2fb-4761-a34c-bd2164eae5b7 None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:44:30.486277 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d5b8d8-b2fb-4761-a34c-bd2164eae5b7 None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:44:30.493834 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab9c7e51-bb59-4ef8-97b9-49fde017585d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:44:30.494565 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9c7e51-bb59-4ef8-97b9-49fde017585d admin admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:44:30.494995 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9c7e51-bb59-4ef8-97b9-49fde017585d admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:44:30.495280 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9c7e51-bb59-4ef8-97b9-49fde017585d admin admin] PATH_INFO: `/v3/domains` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:44:30.496028 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab9c7e51-bb59-4ef8-97b9-49fde017585d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:44:30.496336 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab9c7e51-bb59-4ef8-97b9-49fde017585d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:44:30.498174 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab9c7e51-bb59-4ef8-97b9-49fde017585d admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:44:30.512862 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1290/2576] 10.222.0.33 () {58 vars in 1152 bytes} [Mon Jul 25 05:44:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:44:30.517258 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc28b34-3d16-4bbf-85e8-2853a39380d1 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:44:30.528631 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc28b34-3d16-4bbf-85e8-2853a39380d1 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:44:30.534371 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a972c5-eb10-417d-8183-ab2247b054ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:44:30.535532 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a972c5-eb10-417d-8183-ab2247b054ec admin admin] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:44:30.535817 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a972c5-eb10-417d-8183-ab2247b054ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:44:30.536089 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a972c5-eb10-417d-8183-ab2247b054ec admin admin] PATH_INFO: `/v3/projects` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:44:30.536987 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36a972c5-eb10-417d-8183-ab2247b054ec admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:44:30.540023 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36a972c5-eb10-417d-8183-ab2247b054ec admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:44:30.587672 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1287/2577] 10.222.0.33 () {60 vars in 1152 bytes} [Mon Jul 25 05:44:30 2022] POST /identity/v3/projects => generated 381 bytes in 71 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 05:44:30.592610 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab9c7e51-bb59-4ef8-97b9-49fde017585d admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:44:30.612710 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab9c7e51-bb59-4ef8-97b9-49fde017585d admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:44:30.668660 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:44:30.703562 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42313728-73e3-4279-91a9-21229ed064db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:44:30.704235 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42313728-73e3-4279-91a9-21229ed064db admin admin] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:44:30.704526 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42313728-73e3-4279-91a9-21229ed064db admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:44:30.704829 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42313728-73e3-4279-91a9-21229ed064db admin admin] PATH_INFO: `/v3/users` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:44:30.705849 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42313728-73e3-4279-91a9-21229ed064db admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:44:30.707830 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42313728-73e3-4279-91a9-21229ed064db admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:44:30.736144 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1291/2578] 10.222.0.33 () {60 vars in 1143 bytes} [Mon Jul 25 05:44:30 2022] POST /identity/v3/users => generated 291 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 05:44:30.740357 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a972c5-eb10-417d-8183-ab2247b054ec admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:44:30.754468 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a972c5-eb10-417d-8183-ab2247b054ec admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:44:30.763430 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd35dbe-16ff-46e1-9931-cbaa41151bc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:44:30.764522 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd35dbe-16ff-46e1-9931-cbaa41151bc7 admin admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:44:30.765684 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd35dbe-16ff-46e1-9931-cbaa41151bc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:44:30.766194 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd35dbe-16ff-46e1-9931-cbaa41151bc7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:44:30.767268 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdd35dbe-16ff-46e1-9931-cbaa41151bc7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:44:30.769579 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdd35dbe-16ff-46e1-9931-cbaa41151bc7 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:44:30.789382 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42313728-73e3-4279-91a9-21229ed064db admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:44:30.790195 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1288/2579] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jul 25 05:44:30 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 25 05:44:30.812149 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42313728-73e3-4279-91a9-21229ed064db admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:44:30.817820 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-031510c3-0227-4a8a-a268-79fd6ab6e94a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:44:30.818874 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031510c3-0227-4a8a-a268-79fd6ab6e94a admin admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:44:30.819887 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031510c3-0227-4a8a-a268-79fd6ab6e94a admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:44:30.820507 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031510c3-0227-4a8a-a268-79fd6ab6e94a admin admin] PATH_INFO: `/v3/roles` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:44:30.822328 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-031510c3-0227-4a8a-a268-79fd6ab6e94a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:44:30.824245 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-031510c3-0227-4a8a-a268-79fd6ab6e94a admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:44:30.843526 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1292/2580] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jul 25 05:44:30 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 25 05:44:30.847419 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd35dbe-16ff-46e1-9931-cbaa41151bc7 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:44:30.861364 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd35dbe-16ff-46e1-9931-cbaa41151bc7 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:44:30.871171 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a6c0fe0-c691-4df9-ad2b-952095254b74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:44:30.872278 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6c0fe0-c691-4df9-ad2b-952095254b74 admin admin] REQUEST_METHOD: `PUT` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:44:30.872664 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6c0fe0-c691-4df9-ad2b-952095254b74 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:44:30.873058 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6c0fe0-c691-4df9-ad2b-952095254b74 admin admin] PATH_INFO: `/v3/projects/500245c3ddb243e0bacd45d01d000d59/users/44b56edb971248f7914d41c6eec0765f/roles/5074e74e35f54d4b9bb415c34999a82a` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:44:30.896049 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a6c0fe0-c691-4df9-ad2b-952095254b74 admin admin] RBAC: Authorizing `identity:create_grant(project_id=500245c3ddb243e0bacd45d01d000d59, user_id=44b56edb971248f7914d41c6eec0765f, role_id=5074e74e35f54d4b9bb415c34999a82a)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:44:30.898668 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a6c0fe0-c691-4df9-ad2b-952095254b74 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:44:30.930403 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1289/2581] 10.222.0.33 () {60 vars in 1482 bytes} [Mon Jul 25 05:44:30 2022] PUT /identity/v3/projects/500245c3ddb243e0bacd45d01d000d59/users/44b56edb971248f7914d41c6eec0765f/roles/5074e74e35f54d4b9bb415c34999a82a => generated 0 bytes in 83 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 05:44:30.946225 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa487e5-1d26-4724-a242-ac80d1b033ab None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:44:30.946225 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa487e5-1d26-4724-a242-ac80d1b033ab None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:44:30.946225 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa487e5-1d26-4724-a242-ac80d1b033ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:44:30.981442 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-faa487e5-1d26-4724-a242-ac80d1b033ab None None] MFA Rules not processed for user `44b56edb971248f7914d41c6eec0765f`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:44:30.982852 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-faa487e5-1d26-4724-a242-ac80d1b033ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:44:31.072151 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1293/2582] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 25 05:44:30 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:44:31.077236 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a6c0fe0-c691-4df9-ad2b-952095254b74 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:44:31.191235 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a6c0fe0-c691-4df9-ad2b-952095254b74 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:44:31.198430 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-970297a8-416d-4494-8edb-fa1584c89223 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:44:31.199604 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970297a8-416d-4494-8edb-fa1584c89223 admin admin] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:44:31.200173 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970297a8-416d-4494-8edb-fa1584c89223 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:44:31.200672 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970297a8-416d-4494-8edb-fa1584c89223 admin admin] PATH_INFO: `/v3/projects` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:44:31.201653 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-970297a8-416d-4494-8edb-fa1584c89223 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:44:31.204020 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-970297a8-416d-4494-8edb-fa1584c89223 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:44:31.259299 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1290/2583] 10.222.0.33 () {60 vars in 1152 bytes} [Mon Jul 25 05:44:31 2022] POST /identity/v3/projects => generated 383 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 05:44:31.262781 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa487e5-1d26-4724-a242-ac80d1b033ab None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:44:31.275170 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa487e5-1d26-4724-a242-ac80d1b033ab None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:44:31.352621 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f67c591f-dd72-4439-b948-4acb0a271a00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:44:31.353798 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67c591f-dd72-4439-b948-4acb0a271a00 admin admin] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:44:31.354286 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67c591f-dd72-4439-b948-4acb0a271a00 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:44:31.354764 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67c591f-dd72-4439-b948-4acb0a271a00 admin admin] PATH_INFO: `/v3/users` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:44:31.355727 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f67c591f-dd72-4439-b948-4acb0a271a00 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:44:31.358204 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f67c591f-dd72-4439-b948-4acb0a271a00 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:44:31.396376 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1294/2584] 10.222.0.33 () {60 vars in 1143 bytes} [Mon Jul 25 05:44:31 2022] POST /identity/v3/users => generated 298 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 25 05:44:31.396999 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-970297a8-416d-4494-8edb-fa1584c89223 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:44:31.408192 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-970297a8-416d-4494-8edb-fa1584c89223 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:44:31.413878 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d356f005-20bc-4b40-92fe-c01534477542 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:44:31.414713 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d356f005-20bc-4b40-92fe-c01534477542 admin admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:44:31.415070 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d356f005-20bc-4b40-92fe-c01534477542 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:44:31.415427 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d356f005-20bc-4b40-92fe-c01534477542 admin admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:44:31.416557 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d356f005-20bc-4b40-92fe-c01534477542 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:44:31.418525 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d356f005-20bc-4b40-92fe-c01534477542 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:44:31.433850 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1291/2585] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jul 25 05:44:31 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 05:44:31.439502 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f67c591f-dd72-4439-b948-4acb0a271a00 admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:44:31.450278 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f67c591f-dd72-4439-b948-4acb0a271a00 admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:44:31.460304 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6334a0d8-0ea3-4e2a-9f81-09fe33b32958 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:44:31.461456 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6334a0d8-0ea3-4e2a-9f81-09fe33b32958 admin admin] REQUEST_METHOD: `PUT` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:44:31.461929 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6334a0d8-0ea3-4e2a-9f81-09fe33b32958 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:44:31.462763 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6334a0d8-0ea3-4e2a-9f81-09fe33b32958 admin admin] PATH_INFO: `/v3/projects/c7fbd7ae1b3a4efcbfae522f78bdcc27/users/156552cfceed4f4eb2c06b4acea2f33d/roles/39e5c666285a42fe8fc2c3afca6260d1` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:44:31.485787 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6334a0d8-0ea3-4e2a-9f81-09fe33b32958 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c7fbd7ae1b3a4efcbfae522f78bdcc27, user_id=156552cfceed4f4eb2c06b4acea2f33d, role_id=39e5c666285a42fe8fc2c3afca6260d1)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:44:31.488226 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6334a0d8-0ea3-4e2a-9f81-09fe33b32958 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:44:31.518817 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1295/2586] 10.222.0.33 () {60 vars in 1482 bytes} [Mon Jul 25 05:44:31 2022] PUT /identity/v3/projects/c7fbd7ae1b3a4efcbfae522f78bdcc27/users/156552cfceed4f4eb2c06b4acea2f33d/roles/39e5c666285a42fe8fc2c3afca6260d1 => generated 0 bytes in 80 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 05:44:31.524911 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d356f005-20bc-4b40-92fe-c01534477542 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:44:31.536517 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d356f005-20bc-4b40-92fe-c01534477542 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:44:31.620074 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4df54f4-5acf-4bb8-9041-aae6dd869bae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:44:31.620728 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4df54f4-5acf-4bb8-9041-aae6dd869bae admin admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:44:31.621204 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4df54f4-5acf-4bb8-9041-aae6dd869bae admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:44:31.621490 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4df54f4-5acf-4bb8-9041-aae6dd869bae admin admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:44:31.622237 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4df54f4-5acf-4bb8-9041-aae6dd869bae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:44:31.623850 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4df54f4-5acf-4bb8-9041-aae6dd869bae admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:44:31.636824 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1292/2587] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jul 25 05:44:31 2022] GET /identity/v3/roles => generated 1420 bytes in 113 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 05:44:31.640200 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6334a0d8-0ea3-4e2a-9f81-09fe33b32958 admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:44:31.650542 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:44:31.657402 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6334a0d8-0ea3-4e2a-9f81-09fe33b32958 admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:44:31.666916 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd884d21-56be-419b-b08d-4cb0d446d8cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:44:31.668016 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd884d21-56be-419b-b08d-4cb0d446d8cc admin admin] REQUEST_METHOD: `PUT` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:44:31.668410 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd884d21-56be-419b-b08d-4cb0d446d8cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:44:31.668810 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd884d21-56be-419b-b08d-4cb0d446d8cc admin admin] PATH_INFO: `/v3/projects/c7fbd7ae1b3a4efcbfae522f78bdcc27/users/156552cfceed4f4eb2c06b4acea2f33d/roles/39e5c666285a42fe8fc2c3afca6260d1` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:44:31.670993 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd884d21-56be-419b-b08d-4cb0d446d8cc admin admin] RBAC: Authorizing `identity:create_grant(project_id=c7fbd7ae1b3a4efcbfae522f78bdcc27, user_id=156552cfceed4f4eb2c06b4acea2f33d, role_id=39e5c666285a42fe8fc2c3afca6260d1)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:44:31.674040 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd884d21-56be-419b-b08d-4cb0d446d8cc admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:44:31.692059 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1296/2588] 10.222.0.33 () {60 vars in 1482 bytes} [Mon Jul 25 05:44:31 2022] PUT /identity/v3/projects/c7fbd7ae1b3a4efcbfae522f78bdcc27/users/156552cfceed4f4eb2c06b4acea2f33d/roles/39e5c666285a42fe8fc2c3afca6260d1 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 05:44:31.762920 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db2d93f-8e69-481e-829a-2d8c990656b5 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:44:31.763506 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db2d93f-8e69-481e-829a-2d8c990656b5 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:44:31.763506 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db2d93f-8e69-481e-829a-2d8c990656b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:44:31.771611 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:44:31.806960 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-3db2d93f-8e69-481e-829a-2d8c990656b5 None None] MFA Rules not processed for user `156552cfceed4f4eb2c06b4acea2f33d`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:44:31.808253 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-3db2d93f-8e69-481e-829a-2d8c990656b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:44:31.933570 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1293/2589] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 25 05:44:31 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 05:44:31.940856 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd884d21-56be-419b-b08d-4cb0d446d8cc admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:44:31.942927 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-cd884d21-56be-419b-b08d-4cb0d446d8cc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:44:32.051456 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd884d21-56be-419b-b08d-4cb0d446d8cc admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:44:32.062035 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e52e855-b556-482e-8ecb-bb30f1572d7d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db57b478c25643dbaf936f90f6e17d0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92afba266cce4d1bad4134c351e74be3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:44:32.062830 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e52e855-b556-482e-8ecb-bb30f1572d7d service cinder] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:44:32.063181 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e52e855-b556-482e-8ecb-bb30f1572d7d service cinder] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:44:32.063535 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e52e855-b556-482e-8ecb-bb30f1572d7d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:44:32.092592 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e52e855-b556-482e-8ecb-bb30f1572d7d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:44:32.094819 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e52e855-b556-482e-8ecb-bb30f1572d7d service cinder] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:44:32.123236 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1297/2590] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jul 25 05:44:31 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 183 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 05:44:32.281051 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db2d93f-8e69-481e-829a-2d8c990656b5 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:44:32.294096 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db2d93f-8e69-481e-829a-2d8c990656b5 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:44:32.309517 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d529eaf4-4fad-40e5-9faf-5edbf1839bbc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db57b478c25643dbaf936f90f6e17d0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92afba266cce4d1bad4134c351e74be3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:44:32.311514 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d529eaf4-4fad-40e5-9faf-5edbf1839bbc service cinder] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:44:32.312448 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d529eaf4-4fad-40e5-9faf-5edbf1839bbc service cinder] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:44:32.312968 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d529eaf4-4fad-40e5-9faf-5edbf1839bbc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:44:32.432465 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d529eaf4-4fad-40e5-9faf-5edbf1839bbc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:44:32.434222 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d529eaf4-4fad-40e5-9faf-5edbf1839bbc service cinder] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:44:32.447872 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1294/2591] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jul 25 05:44:32 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 168 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 05:44:32.860357 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e52e855-b556-482e-8ecb-bb30f1572d7d service cinder] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:44:33.012715 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e52e855-b556-482e-8ecb-bb30f1572d7d service cinder] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:44:33.018545 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9084e928-5a56-48e5-a52b-11d0dd0d56a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:44:33.019501 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9084e928-5a56-48e5-a52b-11d0dd0d56a0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:44:33.019861 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9084e928-5a56-48e5-a52b-11d0dd0d56a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:44:33.020239 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9084e928-5a56-48e5-a52b-11d0dd0d56a0 admin admin] PATH_INFO: `/v3/users/44b56edb971248f7914d41c6eec0765f` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:44:33.021603 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9084e928-5a56-48e5-a52b-11d0dd0d56a0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=44b56edb971248f7914d41c6eec0765f)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:44:33.024684 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9084e928-5a56-48e5-a52b-11d0dd0d56a0 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:44:33.120195 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-9084e928-5a56-48e5-a52b-11d0dd0d56a0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '44b56edb971248f7914d41c6eec0765f'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:44:33.142345 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-9084e928-5a56-48e5-a52b-11d0dd0d56a0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '44b56edb971248f7914d41c6eec0765f'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:44:33.143095 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-9084e928-5a56-48e5-a52b-11d0dd0d56a0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '44b56edb971248f7914d41c6eec0765f'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:44:33.154150 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-9084e928-5a56-48e5-a52b-11d0dd0d56a0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '44b56edb971248f7914d41c6eec0765f'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:44:33.154841 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-9084e928-5a56-48e5-a52b-11d0dd0d56a0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '44b56edb971248f7914d41c6eec0765f'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:44:33.178106 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1298/2592] 10.222.0.33 () {58 vars in 1223 bytes} [Mon Jul 25 05:44:32 2022] DELETE /identity/v3/users/44b56edb971248f7914d41c6eec0765f => generated 0 bytes in 318 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 05:44:33.183257 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da7afc1-f768-46d9-b307-de949a8b9933 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:44:33.184342 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da7afc1-f768-46d9-b307-de949a8b9933 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:44:33.184930 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da7afc1-f768-46d9-b307-de949a8b9933 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:44:33.213781 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-4da7afc1-f768-46d9-b307-de949a8b9933 None None] MFA Rules not processed for user `1eae540242d94c10b0ad33960a855f08`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:44:33.215858 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-4da7afc1-f768-46d9-b307-de949a8b9933 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:44:33.295026 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1295/2593] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 25 05:44:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:44:33.307168 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9084e928-5a56-48e5-a52b-11d0dd0d56a0 admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:44:33.309316 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-9084e928-5a56-48e5-a52b-11d0dd0d56a0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:44:33.358141 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:44:33.450215 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9084e928-5a56-48e5-a52b-11d0dd0d56a0 admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:44:33.455726 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e717ff-2f4d-4003-acf4-8a8fc198a401 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b2931200f99e43d79a50a9f6e02738e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92afba266cce4d1bad4134c351e74be3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:44:33.456945 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e717ff-2f4d-4003-acf4-8a8fc198a401 service neutron] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:44:33.457289 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e717ff-2f4d-4003-acf4-8a8fc198a401 service neutron] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:44:33.457594 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e717ff-2f4d-4003-acf4-8a8fc198a401 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:44:33.472046 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02e717ff-2f4d-4003-acf4-8a8fc198a401 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:44:33.474738 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02e717ff-2f4d-4003-acf4-8a8fc198a401 service neutron] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:44:33.499831 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1299/2594] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jul 25 05:44:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 193 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 05:44:34.254769 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da7afc1-f768-46d9-b307-de949a8b9933 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:44:34.259451 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-4da7afc1-f768-46d9-b307-de949a8b9933 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:44:34.285024 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da7afc1-f768-46d9-b307-de949a8b9933 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:44:34.307051 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-390cd9ea-3311-49a0-a113-4cd068966dfc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:44:34.308191 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390cd9ea-3311-49a0-a113-4cd068966dfc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:44:34.310604 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390cd9ea-3311-49a0-a113-4cd068966dfc admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:44:34.311062 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390cd9ea-3311-49a0-a113-4cd068966dfc admin admin] PATH_INFO: `/v3/projects/500245c3ddb243e0bacd45d01d000d59` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:44:34.313217 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-390cd9ea-3311-49a0-a113-4cd068966dfc admin admin] RBAC: Authorizing `identity:delete_project(project_id=500245c3ddb243e0bacd45d01d000d59)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:44:34.316448 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-390cd9ea-3311-49a0-a113-4cd068966dfc admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:44:34.372538 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:44:34.418982 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-390cd9ea-3311-49a0-a113-4cd068966dfc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '500245c3ddb243e0bacd45d01d000d59', 'request_id': 'req-390cd9ea-3311-49a0-a113-4cd068966dfc'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:44:34.425685 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-390cd9ea-3311-49a0-a113-4cd068966dfc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '500245c3ddb243e0bacd45d01d000d59', 'request_id': 'req-390cd9ea-3311-49a0-a113-4cd068966dfc'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:44:34.434153 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-390cd9ea-3311-49a0-a113-4cd068966dfc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '500245c3ddb243e0bacd45d01d000d59', 'request_id': 'req-390cd9ea-3311-49a0-a113-4cd068966dfc'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:44:34.453972 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-390cd9ea-3311-49a0-a113-4cd068966dfc admin admin] The token cache is being invalidate because project 500245c3ddb243e0bacd45d01d000d59 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80249) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 05:44:34.454398 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-390cd9ea-3311-49a0-a113-4cd068966dfc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:44:34.455030 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-390cd9ea-3311-49a0-a113-4cd068966dfc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:44:34.460397 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1296/2595] 10.222.0.33 () {58 vars in 1232 bytes} [Mon Jul 25 05:44:34 2022] DELETE /identity/v3/projects/500245c3ddb243e0bacd45d01d000d59 => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 05:44:34.461651 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e717ff-2f4d-4003-acf4-8a8fc198a401 service neutron] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:44:34.474059 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e717ff-2f4d-4003-acf4-8a8fc198a401 service neutron] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:44:34.546754 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a30f3a-ca4e-4183-b44a-13e3d925f94e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:44:34.548013 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a30f3a-ca4e-4183-b44a-13e3d925f94e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:44:34.548518 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a30f3a-ca4e-4183-b44a-13e3d925f94e admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:44:34.548991 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a30f3a-ca4e-4183-b44a-13e3d925f94e admin admin] PATH_INFO: `/v3/users/156552cfceed4f4eb2c06b4acea2f33d` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:44:34.550451 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87a30f3a-ca4e-4183-b44a-13e3d925f94e admin admin] RBAC: Authorizing `identity:delete_user(user_id=156552cfceed4f4eb2c06b4acea2f33d)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:44:34.552735 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87a30f3a-ca4e-4183-b44a-13e3d925f94e admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:44:34.644030 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-87a30f3a-ca4e-4183-b44a-13e3d925f94e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '156552cfceed4f4eb2c06b4acea2f33d'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:44:34.666696 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-87a30f3a-ca4e-4183-b44a-13e3d925f94e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '156552cfceed4f4eb2c06b4acea2f33d'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:44:34.667555 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-87a30f3a-ca4e-4183-b44a-13e3d925f94e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '156552cfceed4f4eb2c06b4acea2f33d'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:44:34.680208 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-87a30f3a-ca4e-4183-b44a-13e3d925f94e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '156552cfceed4f4eb2c06b4acea2f33d'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:44:34.681600 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-87a30f3a-ca4e-4183-b44a-13e3d925f94e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '156552cfceed4f4eb2c06b4acea2f33d'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:44:34.708540 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1300/2596] 10.222.0.33 () {58 vars in 1223 bytes} [Mon Jul 25 05:44:34 2022] DELETE /identity/v3/users/156552cfceed4f4eb2c06b4acea2f33d => generated 0 bytes in 247 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 05:44:34.834245 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-390cd9ea-3311-49a0-a113-4cd068966dfc admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:44:34.838312 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-390cd9ea-3311-49a0-a113-4cd068966dfc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:44:34.865527 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-390cd9ea-3311-49a0-a113-4cd068966dfc admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:44:34.953300 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d22ea759-da22-4550-8453-9b6f367df1ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:44:34.954457 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22ea759-da22-4550-8453-9b6f367df1ed admin admin] REQUEST_METHOD: `DELETE` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:44:34.954862 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22ea759-da22-4550-8453-9b6f367df1ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:44:34.955276 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22ea759-da22-4550-8453-9b6f367df1ed admin admin] PATH_INFO: `/v3/users/c1ae82a1862048cb9038b7c76190da8a` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:44:34.956629 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d22ea759-da22-4550-8453-9b6f367df1ed admin admin] RBAC: Authorizing `identity:delete_user(user_id=c1ae82a1862048cb9038b7c76190da8a)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:44:34.958708 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d22ea759-da22-4550-8453-9b6f367df1ed admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:44:35.048961 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-d22ea759-da22-4550-8453-9b6f367df1ed admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c1ae82a1862048cb9038b7c76190da8a'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:44:35.050578 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-d22ea759-da22-4550-8453-9b6f367df1ed admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c1ae82a1862048cb9038b7c76190da8a'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:44:35.061037 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-d22ea759-da22-4550-8453-9b6f367df1ed admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c1ae82a1862048cb9038b7c76190da8a'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:44:35.078556 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-d22ea759-da22-4550-8453-9b6f367df1ed admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c1ae82a1862048cb9038b7c76190da8a'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:44:35.079313 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-d22ea759-da22-4550-8453-9b6f367df1ed admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c1ae82a1862048cb9038b7c76190da8a'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:44:35.096340 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1297/2597] 10.222.0.33 () {58 vars in 1223 bytes} [Mon Jul 25 05:44:34 2022] DELETE /identity/v3/users/c1ae82a1862048cb9038b7c76190da8a => generated 0 bytes in 263 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 05:44:35.187456 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a30f3a-ca4e-4183-b44a-13e3d925f94e admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:44:35.187849 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-87a30f3a-ca4e-4183-b44a-13e3d925f94e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:44:35.207347 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a30f3a-ca4e-4183-b44a-13e3d925f94e admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:44:35.262051 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:44:35.292988 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b99f56f3-16f6-47c2-be87-cbcbc7a166e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:44:35.293941 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99f56f3-16f6-47c2-be87-cbcbc7a166e8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:44:35.294298 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99f56f3-16f6-47c2-be87-cbcbc7a166e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:44:35.295078 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99f56f3-16f6-47c2-be87-cbcbc7a166e8 admin admin] PATH_INFO: `/v3/projects/c7fbd7ae1b3a4efcbfae522f78bdcc27` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:44:35.295999 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b99f56f3-16f6-47c2-be87-cbcbc7a166e8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c7fbd7ae1b3a4efcbfae522f78bdcc27)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:44:35.298250 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b99f56f3-16f6-47c2-be87-cbcbc7a166e8 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:44:35.363007 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-b99f56f3-16f6-47c2-be87-cbcbc7a166e8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c7fbd7ae1b3a4efcbfae522f78bdcc27', 'request_id': 'req-b99f56f3-16f6-47c2-be87-cbcbc7a166e8'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:44:35.371865 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d22ea759-da22-4550-8453-9b6f367df1ed admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:44:35.374889 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-d22ea759-da22-4550-8453-9b6f367df1ed admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:44:35.375028 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-b99f56f3-16f6-47c2-be87-cbcbc7a166e8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c7fbd7ae1b3a4efcbfae522f78bdcc27', 'request_id': 'req-b99f56f3-16f6-47c2-be87-cbcbc7a166e8'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:44:35.391206 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-b99f56f3-16f6-47c2-be87-cbcbc7a166e8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c7fbd7ae1b3a4efcbfae522f78bdcc27', 'request_id': 'req-b99f56f3-16f6-47c2-be87-cbcbc7a166e8'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:44:35.401730 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d22ea759-da22-4550-8453-9b6f367df1ed admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:44:35.412358 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-b99f56f3-16f6-47c2-be87-cbcbc7a166e8 admin admin] The token cache is being invalidate because project c7fbd7ae1b3a4efcbfae522f78bdcc27 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80248) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 05:44:35.412689 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-b99f56f3-16f6-47c2-be87-cbcbc7a166e8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:44:35.413990 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-b99f56f3-16f6-47c2-be87-cbcbc7a166e8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:44:35.415590 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1301/2598] 10.222.0.33 () {58 vars in 1232 bytes} [Mon Jul 25 05:44:35 2022] DELETE /identity/v3/projects/c7fbd7ae1b3a4efcbfae522f78bdcc27 => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 05:44:35.420889 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4cfbb23-2c6d-40e8-a3c3-5c7d343e2d5b None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:44:35.421443 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4cfbb23-2c6d-40e8-a3c3-5c7d343e2d5b None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:44:35.421826 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4cfbb23-2c6d-40e8-a3c3-5c7d343e2d5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:44:35.443304 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:DELETE=17,keystone:SELECT=81,keystone:INSERT=2 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:44:35.444726 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-a4cfbb23-2c6d-40e8-a3c3-5c7d343e2d5b None None] MFA Rules not processed for user `1eae540242d94c10b0ad33960a855f08`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:44:35.446088 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-a4cfbb23-2c6d-40e8-a3c3-5c7d343e2d5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:44:35.486583 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcad5c1f-d235-4731-984d-39bb57c32a7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:44:35.487891 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcad5c1f-d235-4731-984d-39bb57c32a7e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:44:35.488345 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcad5c1f-d235-4731-984d-39bb57c32a7e admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:44:35.488944 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcad5c1f-d235-4731-984d-39bb57c32a7e admin admin] PATH_INFO: `/v3/projects/f0b7d41e75df46f28cebff70c16d4750` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:44:35.490436 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcad5c1f-d235-4731-984d-39bb57c32a7e admin admin] RBAC: Authorizing `identity:delete_project(project_id=f0b7d41e75df46f28cebff70c16d4750)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:44:35.493444 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcad5c1f-d235-4731-984d-39bb57c32a7e admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:44:35.529070 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1302/2599] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 25 05:44:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 05:44:35.536675 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ddad844-8abd-4164-9abb-21d6d2f1d1fe None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:44:35.537126 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ddad844-8abd-4164-9abb-21d6d2f1d1fe None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:44:35.537487 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ddad844-8abd-4164-9abb-21d6d2f1d1fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:44:35.560323 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-6ddad844-8abd-4164-9abb-21d6d2f1d1fe None None] MFA Rules not processed for user `1eae540242d94c10b0ad33960a855f08`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:44:35.561666 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-6ddad844-8abd-4164-9abb-21d6d2f1d1fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:44:35.562140 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-fcad5c1f-d235-4731-984d-39bb57c32a7e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f0b7d41e75df46f28cebff70c16d4750', 'request_id': 'req-fcad5c1f-d235-4731-984d-39bb57c32a7e'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:44:35.568742 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-fcad5c1f-d235-4731-984d-39bb57c32a7e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f0b7d41e75df46f28cebff70c16d4750', 'request_id': 'req-fcad5c1f-d235-4731-984d-39bb57c32a7e'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:44:35.577134 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-fcad5c1f-d235-4731-984d-39bb57c32a7e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f0b7d41e75df46f28cebff70c16d4750', 'request_id': 'req-fcad5c1f-d235-4731-984d-39bb57c32a7e'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:44:35.595177 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-fcad5c1f-d235-4731-984d-39bb57c32a7e admin admin] The token cache is being invalidate because project f0b7d41e75df46f28cebff70c16d4750 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80249) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 05:44:35.595736 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-fcad5c1f-d235-4731-984d-39bb57c32a7e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:44:35.596618 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-fcad5c1f-d235-4731-984d-39bb57c32a7e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:44:35.602885 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1298/2600] 10.222.0.33 () {58 vars in 1232 bytes} [Mon Jul 25 05:44:35 2022] DELETE /identity/v3/projects/f0b7d41e75df46f28cebff70c16d4750 => generated 0 bytes in 228 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 25 05:44:35.605644 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d628992-08d9-4493-83fc-36d717ea649e None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:44:35.605644 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d628992-08d9-4493-83fc-36d717ea649e None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:44:35.607342 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d628992-08d9-4493-83fc-36d717ea649e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:44:35.627515 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-4d628992-08d9-4493-83fc-36d717ea649e None None] MFA Rules not processed for user `1eae540242d94c10b0ad33960a855f08`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:44:35.629249 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-4d628992-08d9-4493-83fc-36d717ea649e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:44:35.646071 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1303/2601] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 25 05:44:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:44:35.650496 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ddad844-8abd-4164-9abb-21d6d2f1d1fe None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:44:35.661906 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ddad844-8abd-4164-9abb-21d6d2f1d1fe None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:44:35.667549 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-695a13e0-f2cb-4250-b138-ad8f9a3914c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:44:35.668217 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695a13e0-f2cb-4250-b138-ad8f9a3914c4 admin admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:44:35.668504 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695a13e0-f2cb-4250-b138-ad8f9a3914c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:44:35.668801 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695a13e0-f2cb-4250-b138-ad8f9a3914c4 admin admin] PATH_INFO: `/v3/domains` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:44:35.669554 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-695a13e0-f2cb-4250-b138-ad8f9a3914c4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:44:35.669863 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-695a13e0-f2cb-4250-b138-ad8f9a3914c4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:44:35.672115 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-695a13e0-f2cb-4250-b138-ad8f9a3914c4 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:44:35.673834 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:44:35.692365 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1304/2602] 10.222.0.33 () {58 vars in 1152 bytes} [Mon Jul 25 05:44:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 05:44:35.696929 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-695a13e0-f2cb-4250-b138-ad8f9a3914c4 admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:44:35.710277 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1299/2603] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 25 05:44:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:44:35.711846 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-695a13e0-f2cb-4250-b138-ad8f9a3914c4 admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:44:35.719714 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc263224-8a92-4a45-ac03-5f96897f3e94 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:44:35.720160 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc263224-8a92-4a45-ac03-5f96897f3e94 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:44:35.720523 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b54eff9f-72cb-4dcd-b39d-052d2ea03773 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:44:35.720649 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc263224-8a92-4a45-ac03-5f96897f3e94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:44:35.721273 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54eff9f-72cb-4dcd-b39d-052d2ea03773 admin admin] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:44:35.721557 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54eff9f-72cb-4dcd-b39d-052d2ea03773 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:44:35.721835 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54eff9f-72cb-4dcd-b39d-052d2ea03773 admin admin] PATH_INFO: `/v3/projects` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:44:35.722531 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b54eff9f-72cb-4dcd-b39d-052d2ea03773 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:44:35.724850 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b54eff9f-72cb-4dcd-b39d-052d2ea03773 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:44:35.750989 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-dc263224-8a92-4a45-ac03-5f96897f3e94 None None] MFA Rules not processed for user `1eae540242d94c10b0ad33960a855f08`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:44:35.753093 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-dc263224-8a92-4a45-ac03-5f96897f3e94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:44:35.766624 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1300/2604] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 25 05:44:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:44:35.768330 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1305/2605] 10.222.0.33 () {60 vars in 1152 bytes} [Mon Jul 25 05:44:35 2022] POST /identity/v3/projects => generated 377 bytes in 72 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 05:44:35.770407 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b54eff9f-72cb-4dcd-b39d-052d2ea03773 admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:44:35.773021 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc263224-8a92-4a45-ac03-5f96897f3e94 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:44:35.783938 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc263224-8a92-4a45-ac03-5f96897f3e94 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:44:35.861677 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:44:35.869774 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b54eff9f-72cb-4dcd-b39d-052d2ea03773 admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:44:35.877343 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-968af69c-370f-4ea7-bb71-aafb23314380 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:44:35.878022 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968af69c-370f-4ea7-bb71-aafb23314380 admin admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:44:35.878314 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968af69c-370f-4ea7-bb71-aafb23314380 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:44:35.878777 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968af69c-370f-4ea7-bb71-aafb23314380 admin admin] PATH_INFO: `/v3/domains` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:44:35.879483 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-968af69c-370f-4ea7-bb71-aafb23314380 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:44:35.879793 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-968af69c-370f-4ea7-bb71-aafb23314380 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:44:35.881968 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-968af69c-370f-4ea7-bb71-aafb23314380 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:44:35.889579 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af63d992-7f36-45a4-b9ca-cf2dfba5d272 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:44:35.890635 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af63d992-7f36-45a4-b9ca-cf2dfba5d272 admin admin] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:44:35.891088 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af63d992-7f36-45a4-b9ca-cf2dfba5d272 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:44:35.892178 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af63d992-7f36-45a4-b9ca-cf2dfba5d272 admin admin] PATH_INFO: `/v3/users` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:44:35.893192 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af63d992-7f36-45a4-b9ca-cf2dfba5d272 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:44:35.896045 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af63d992-7f36-45a4-b9ca-cf2dfba5d272 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:44:35.899870 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1306/2606] 10.222.0.33 () {58 vars in 1152 bytes} [Mon Jul 25 05:44:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 131 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:44:35.906059 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-968af69c-370f-4ea7-bb71-aafb23314380 admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:44:35.922233 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-968af69c-370f-4ea7-bb71-aafb23314380 admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:44:35.928839 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1301/2607] 10.222.0.33 () {60 vars in 1143 bytes} [Mon Jul 25 05:44:35 2022] POST /identity/v3/users => generated 289 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 05:44:35.929716 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c1190e3-f428-4f9e-8af9-21b3f65940b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:44:35.930400 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1190e3-f428-4f9e-8af9-21b3f65940b5 admin admin] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:44:35.930749 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1190e3-f428-4f9e-8af9-21b3f65940b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:44:35.931037 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1190e3-f428-4f9e-8af9-21b3f65940b5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:44:35.931802 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c1190e3-f428-4f9e-8af9-21b3f65940b5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:44:35.932619 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af63d992-7f36-45a4-b9ca-cf2dfba5d272 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:44:35.934166 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c1190e3-f428-4f9e-8af9-21b3f65940b5 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:44:35.949339 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af63d992-7f36-45a4-b9ca-cf2dfba5d272 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:44:35.955652 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cffe3367-a7c4-4b68-850f-f7478e34862d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:44:35.957025 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cffe3367-a7c4-4b68-850f-f7478e34862d admin admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:44:35.957488 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cffe3367-a7c4-4b68-850f-f7478e34862d admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:44:35.958004 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cffe3367-a7c4-4b68-850f-f7478e34862d admin admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:44:35.958933 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cffe3367-a7c4-4b68-850f-f7478e34862d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:44:35.962553 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cffe3367-a7c4-4b68-850f-f7478e34862d admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:44:35.980836 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1302/2608] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jul 25 05:44:35 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 25 05:44:35.984176 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cffe3367-a7c4-4b68-850f-f7478e34862d admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:44:35.988057 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1307/2609] 10.222.0.33 () {60 vars in 1152 bytes} [Mon Jul 25 05:44:35 2022] POST /identity/v3/projects => generated 379 bytes in 83 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 05:44:35.993730 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c1190e3-f428-4f9e-8af9-21b3f65940b5 admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:44:35.997980 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cffe3367-a7c4-4b68-850f-f7478e34862d admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:44:36.077234 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0568f5ea-946c-4207-8928-4bf10217b899 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:44:36.078257 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0568f5ea-946c-4207-8928-4bf10217b899 admin admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:44:36.078736 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0568f5ea-946c-4207-8928-4bf10217b899 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:44:36.079192 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0568f5ea-946c-4207-8928-4bf10217b899 admin admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:44:36.080095 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0568f5ea-946c-4207-8928-4bf10217b899 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:44:36.082081 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0568f5ea-946c-4207-8928-4bf10217b899 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:44:36.093944 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1303/2610] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jul 25 05:44:35 2022] GET /identity/v3/roles => generated 1420 bytes in 110 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 05:44:36.098626 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0568f5ea-946c-4207-8928-4bf10217b899 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:44:36.117286 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0568f5ea-946c-4207-8928-4bf10217b899 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:44:36.118205 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c1190e3-f428-4f9e-8af9-21b3f65940b5 admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:44:36.122867 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec46b07-e8f7-46a8-bc0b-6f4d8397d5b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:44:36.123848 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec46b07-e8f7-46a8-bc0b-6f4d8397d5b7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:44:36.124129 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec46b07-e8f7-46a8-bc0b-6f4d8397d5b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:44:36.124414 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec46b07-e8f7-46a8-bc0b-6f4d8397d5b7 admin admin] PATH_INFO: `/v3/projects/ddeae234b3b04f50ba2fbbf0b5043de0/users/ad83ed1e819346f3ab0de83c8cb83278/roles/5074e74e35f54d4b9bb415c34999a82a` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:44:36.125408 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-802e354e-163a-468c-9703-c5c343b76a6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:44:36.126251 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802e354e-163a-468c-9703-c5c343b76a6e admin admin] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:44:36.126623 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802e354e-163a-468c-9703-c5c343b76a6e admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:44:36.128155 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802e354e-163a-468c-9703-c5c343b76a6e admin admin] PATH_INFO: `/v3/users` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:44:36.128155 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-802e354e-163a-468c-9703-c5c343b76a6e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:44:36.130020 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-802e354e-163a-468c-9703-c5c343b76a6e admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:44:36.146621 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ec46b07-e8f7-46a8-bc0b-6f4d8397d5b7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ddeae234b3b04f50ba2fbbf0b5043de0, user_id=ad83ed1e819346f3ab0de83c8cb83278, role_id=5074e74e35f54d4b9bb415c34999a82a)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:44:36.153541 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ec46b07-e8f7-46a8-bc0b-6f4d8397d5b7 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:44:36.161643 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1308/2611] 10.222.0.33 () {60 vars in 1143 bytes} [Mon Jul 25 05:44:35 2022] POST /identity/v3/users => generated 290 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 05:44:36.163078 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-802e354e-163a-468c-9703-c5c343b76a6e admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:44:36.180836 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1304/2612] 10.222.0.33 () {60 vars in 1482 bytes} [Mon Jul 25 05:44:36 2022] PUT /identity/v3/projects/ddeae234b3b04f50ba2fbbf0b5043de0/users/ad83ed1e819346f3ab0de83c8cb83278/roles/5074e74e35f54d4b9bb415c34999a82a => generated 0 bytes in 74 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 05:44:36.181059 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-802e354e-163a-468c-9703-c5c343b76a6e admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:44:36.187699 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507592e7-ab63-498b-be15-c7bee27e40f0 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:44:36.188075 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507592e7-ab63-498b-be15-c7bee27e40f0 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:44:36.188422 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507592e7-ab63-498b-be15-c7bee27e40f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:44:36.223329 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:44:36.224234 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-507592e7-ab63-498b-be15-c7bee27e40f0 None None] MFA Rules not processed for user `ad83ed1e819346f3ab0de83c8cb83278`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:44:36.226500 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-507592e7-ab63-498b-be15-c7bee27e40f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:44:36.271473 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27427ee0-1622-4688-b62c-5f813d7762cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:44:36.272684 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27427ee0-1622-4688-b62c-5f813d7762cf admin admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:44:36.273016 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27427ee0-1622-4688-b62c-5f813d7762cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:44:36.273256 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27427ee0-1622-4688-b62c-5f813d7762cf admin admin] PATH_INFO: `/v3/roles` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:44:36.274126 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27427ee0-1622-4688-b62c-5f813d7762cf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:44:36.275862 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27427ee0-1622-4688-b62c-5f813d7762cf admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:44:36.294722 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1309/2613] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jul 25 05:44:36 2022] GET /identity/v3/roles => generated 1420 bytes in 130 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 05:44:36.296196 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27427ee0-1622-4688-b62c-5f813d7762cf admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:44:36.308762 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27427ee0-1622-4688-b62c-5f813d7762cf admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:44:36.313563 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de77a1f-4e60-4f01-aeaa-43b2928796b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:44:36.314234 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de77a1f-4e60-4f01-aeaa-43b2928796b2 admin admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:44:36.314869 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de77a1f-4e60-4f01-aeaa-43b2928796b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:44:36.315155 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de77a1f-4e60-4f01-aeaa-43b2928796b2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:44:36.316365 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4de77a1f-4e60-4f01-aeaa-43b2928796b2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:44:36.318947 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4de77a1f-4e60-4f01-aeaa-43b2928796b2 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:44:36.324694 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1305/2614] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 25 05:44:36 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:44:36.330851 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1310/2615] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jul 25 05:44:36 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 05:44:36.334478 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-507592e7-ab63-498b-be15-c7bee27e40f0 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:44:36.338642 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de77a1f-4e60-4f01-aeaa-43b2928796b2 admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:44:36.340349 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-4de77a1f-4e60-4f01-aeaa-43b2928796b2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:44:36.346398 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-507592e7-ab63-498b-be15-c7bee27e40f0 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:44:36.355275 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0371c885-4579-4ef3-916e-0f9228ce3f62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:44:36.356327 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0371c885-4579-4ef3-916e-0f9228ce3f62 admin admin] REQUEST_METHOD: `PUT` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:44:36.356729 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0371c885-4579-4ef3-916e-0f9228ce3f62 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:44:36.357147 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0371c885-4579-4ef3-916e-0f9228ce3f62 admin admin] PATH_INFO: `/v3/projects/04ff486cc2c14efc954afcb2465c8194/users/217c3c2b10d14adb8e94a2840cde8188/roles/5074e74e35f54d4b9bb415c34999a82a` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:44:36.375661 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0371c885-4579-4ef3-916e-0f9228ce3f62 admin admin] RBAC: Authorizing `identity:create_grant(project_id=04ff486cc2c14efc954afcb2465c8194, user_id=217c3c2b10d14adb8e94a2840cde8188, role_id=5074e74e35f54d4b9bb415c34999a82a)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:44:36.378608 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0371c885-4579-4ef3-916e-0f9228ce3f62 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:44:36.396374 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1306/2616] 10.222.0.33 () {60 vars in 1482 bytes} [Mon Jul 25 05:44:36 2022] PUT /identity/v3/projects/04ff486cc2c14efc954afcb2465c8194/users/217c3c2b10d14adb8e94a2840cde8188/roles/5074e74e35f54d4b9bb415c34999a82a => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 05:44:36.407576 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb94ea8-3b9f-4fd4-84be-4434d0aa4ab1 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:44:36.407917 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb94ea8-3b9f-4fd4-84be-4434d0aa4ab1 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:44:36.408143 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb94ea8-3b9f-4fd4-84be-4434d0aa4ab1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:44:36.415494 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de77a1f-4e60-4f01-aeaa-43b2928796b2 admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:44:36.430096 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:44:36.445887 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-abb94ea8-3b9f-4fd4-84be-4434d0aa4ab1 None None] MFA Rules not processed for user `217c3c2b10d14adb8e94a2840cde8188`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:44:36.447797 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-abb94ea8-3b9f-4fd4-84be-4434d0aa4ab1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:44:36.498672 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-107384dc-7a0f-4463-8101-64517106cbd1 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de99e44bdf9c4030adbe9c0efa3831cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92afba266cce4d1bad4134c351e74be3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:44:36.500959 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107384dc-7a0f-4463-8101-64517106cbd1 service glance] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:44:36.501636 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107384dc-7a0f-4463-8101-64517106cbd1 service glance] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:44:36.501993 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107384dc-7a0f-4463-8101-64517106cbd1 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:44:36.523386 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1307/2617] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 25 05:44:36 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 05:44:36.528104 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb94ea8-3b9f-4fd4-84be-4434d0aa4ab1 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:44:36.581349 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-107384dc-7a0f-4463-8101-64517106cbd1 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:44:36.582917 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-107384dc-7a0f-4463-8101-64517106cbd1 service glance] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:44:36.593937 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1311/2618] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jul 25 05:44:36 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 256 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 25 05:44:36.628975 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb94ea8-3b9f-4fd4-84be-4434d0aa4ab1 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:44:36.638847 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bbbef8d-23b5-4418-9039-11b5c51f47f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:44:36.639819 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bbbef8d-23b5-4418-9039-11b5c51f47f6 admin admin] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:44:36.640430 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bbbef8d-23b5-4418-9039-11b5c51f47f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:44:36.640800 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bbbef8d-23b5-4418-9039-11b5c51f47f6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:44:36.641582 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bbbef8d-23b5-4418-9039-11b5c51f47f6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:44:36.644117 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bbbef8d-23b5-4418-9039-11b5c51f47f6 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:44:36.684631 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1308/2619] 10.222.0.33 () {60 vars in 1152 bytes} [Mon Jul 25 05:44:36 2022] POST /identity/v3/projects => generated 379 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 05:44:36.689428 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-107384dc-7a0f-4463-8101-64517106cbd1 service glance] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:44:36.706812 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-107384dc-7a0f-4463-8101-64517106cbd1 service glance] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:44:36.747502 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:44:36.788421 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf49fa4-f301-4f65-a59c-eddeb0fa0751 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:44:36.790343 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf49fa4-f301-4f65-a59c-eddeb0fa0751 admin admin] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:44:36.790829 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf49fa4-f301-4f65-a59c-eddeb0fa0751 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:44:36.791454 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf49fa4-f301-4f65-a59c-eddeb0fa0751 admin admin] PATH_INFO: `/v3/users` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:44:36.792563 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bf49fa4-f301-4f65-a59c-eddeb0fa0751 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:44:36.794703 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bf49fa4-f301-4f65-a59c-eddeb0fa0751 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:44:36.821086 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1312/2620] 10.222.0.33 () {60 vars in 1143 bytes} [Mon Jul 25 05:44:36 2022] POST /identity/v3/users => generated 296 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 05:44:36.824951 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bbbef8d-23b5-4418-9039-11b5c51f47f6 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:44:36.841771 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bbbef8d-23b5-4418-9039-11b5c51f47f6 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:44:36.846767 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b13dd936-9757-4ef9-b88a-7e022eafb985 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:44:36.847910 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13dd936-9757-4ef9-b88a-7e022eafb985 admin admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:44:36.848349 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13dd936-9757-4ef9-b88a-7e022eafb985 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:44:36.848733 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13dd936-9757-4ef9-b88a-7e022eafb985 admin admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:44:36.850132 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b13dd936-9757-4ef9-b88a-7e022eafb985 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:44:36.853445 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b13dd936-9757-4ef9-b88a-7e022eafb985 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:44:36.868818 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1309/2621] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jul 25 05:44:36 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 25 05:44:36.872986 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf49fa4-f301-4f65-a59c-eddeb0fa0751 admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:44:36.887602 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf49fa4-f301-4f65-a59c-eddeb0fa0751 admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:44:36.893921 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d81cafff-2fc3-4f5f-8df1-7b13e61a0af7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:44:36.894961 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81cafff-2fc3-4f5f-8df1-7b13e61a0af7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:44:36.895418 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81cafff-2fc3-4f5f-8df1-7b13e61a0af7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:44:36.895711 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81cafff-2fc3-4f5f-8df1-7b13e61a0af7 admin admin] PATH_INFO: `/v3/projects/0f6d934558b141c7880c0523cb9767c5/users/100356cea3244b0591d174892ba0d132/roles/39e5c666285a42fe8fc2c3afca6260d1` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:44:36.915474 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d81cafff-2fc3-4f5f-8df1-7b13e61a0af7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0f6d934558b141c7880c0523cb9767c5, user_id=100356cea3244b0591d174892ba0d132, role_id=39e5c666285a42fe8fc2c3afca6260d1)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:44:36.918414 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d81cafff-2fc3-4f5f-8df1-7b13e61a0af7 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:44:36.938237 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1313/2622] 10.222.0.33 () {60 vars in 1482 bytes} [Mon Jul 25 05:44:36 2022] PUT /identity/v3/projects/0f6d934558b141c7880c0523cb9767c5/users/100356cea3244b0591d174892ba0d132/roles/39e5c666285a42fe8fc2c3afca6260d1 => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 05:44:36.942323 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b13dd936-9757-4ef9-b88a-7e022eafb985 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:44:36.953747 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b13dd936-9757-4ef9-b88a-7e022eafb985 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:44:37.025352 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b6b4b1-4ba0-406a-b1bb-4c8b90e86821 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:44:37.026084 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b6b4b1-4ba0-406a-b1bb-4c8b90e86821 admin admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:44:37.026362 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b6b4b1-4ba0-406a-b1bb-4c8b90e86821 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:44:37.026604 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b6b4b1-4ba0-406a-b1bb-4c8b90e86821 admin admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:44:37.027520 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57b6b4b1-4ba0-406a-b1bb-4c8b90e86821 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:44:37.029166 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57b6b4b1-4ba0-406a-b1bb-4c8b90e86821 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:44:37.044660 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1310/2623] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jul 25 05:44:36 2022] GET /identity/v3/roles => generated 1420 bytes in 103 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 25 05:44:37.046012 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d81cafff-2fc3-4f5f-8df1-7b13e61a0af7 admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:44:37.058232 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d81cafff-2fc3-4f5f-8df1-7b13e61a0af7 admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:44:37.063652 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f9dc2a-50cc-4b34-b315-d4b267986854 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:44:37.064918 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f9dc2a-50cc-4b34-b315-d4b267986854 admin admin] REQUEST_METHOD: `PUT` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:44:37.065386 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f9dc2a-50cc-4b34-b315-d4b267986854 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:44:37.065819 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f9dc2a-50cc-4b34-b315-d4b267986854 admin admin] PATH_INFO: `/v3/projects/0f6d934558b141c7880c0523cb9767c5/users/100356cea3244b0591d174892ba0d132/roles/39e5c666285a42fe8fc2c3afca6260d1` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:44:37.079516 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8f9dc2a-50cc-4b34-b315-d4b267986854 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0f6d934558b141c7880c0523cb9767c5, user_id=100356cea3244b0591d174892ba0d132, role_id=39e5c666285a42fe8fc2c3afca6260d1)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:44:37.082217 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8f9dc2a-50cc-4b34-b315-d4b267986854 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:44:37.123526 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1314/2624] 10.222.0.33 () {60 vars in 1482 bytes} [Mon Jul 25 05:44:37 2022] PUT /identity/v3/projects/0f6d934558b141c7880c0523cb9767c5/users/100356cea3244b0591d174892ba0d132/roles/39e5c666285a42fe8fc2c3afca6260d1 => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 25 05:44:37.126909 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3022d0af-c874-4bac-aed9-eb1048caa206 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:44:37.127760 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3022d0af-c874-4bac-aed9-eb1048caa206 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:44:37.127978 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3022d0af-c874-4bac-aed9-eb1048caa206 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:44:37.177903 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:44:37.209495 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-3022d0af-c874-4bac-aed9-eb1048caa206 None None] MFA Rules not processed for user `100356cea3244b0591d174892ba0d132`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:44:37.211319 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-3022d0af-c874-4bac-aed9-eb1048caa206 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:44:37.356703 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1311/2625] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 25 05:44:37 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 231 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:44:37.369241 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f9dc2a-50cc-4b34-b315-d4b267986854 admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:44:37.371057 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-c8f9dc2a-50cc-4b34-b315-d4b267986854 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:44:37.441980 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f9dc2a-50cc-4b34-b315-d4b267986854 admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:44:37.446877 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c911ca82-f3d7-4a51-a2ca-9f0dc4317b50 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db57b478c25643dbaf936f90f6e17d0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92afba266cce4d1bad4134c351e74be3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:44:37.447547 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c911ca82-f3d7-4a51-a2ca-9f0dc4317b50 service cinder] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:44:37.447835 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c911ca82-f3d7-4a51-a2ca-9f0dc4317b50 service cinder] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:44:37.448113 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c911ca82-f3d7-4a51-a2ca-9f0dc4317b50 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:44:37.461015 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c911ca82-f3d7-4a51-a2ca-9f0dc4317b50 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:44:37.462487 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c911ca82-f3d7-4a51-a2ca-9f0dc4317b50 service cinder] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:44:37.475851 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1315/2626] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jul 25 05:44:37 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 05:44:37.570344 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3022d0af-c874-4bac-aed9-eb1048caa206 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:44:37.590624 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3022d0af-c874-4bac-aed9-eb1048caa206 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:44:37.595320 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2dd7290-de24-4275-9180-e7c8175d4dc4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db57b478c25643dbaf936f90f6e17d0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92afba266cce4d1bad4134c351e74be3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:44:37.596216 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2dd7290-de24-4275-9180-e7c8175d4dc4 service cinder] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:44:37.596574 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2dd7290-de24-4275-9180-e7c8175d4dc4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:44:37.597483 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2dd7290-de24-4275-9180-e7c8175d4dc4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:44:37.682704 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2dd7290-de24-4275-9180-e7c8175d4dc4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:44:37.684450 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2dd7290-de24-4275-9180-e7c8175d4dc4 service cinder] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:44:37.706953 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1312/2627] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jul 25 05:44:37 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 25 05:44:45.092051 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b398ad-54b3-4eb8-a9e5-22c05d7b8bb5 None None] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:44:45.092602 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b398ad-54b3-4eb8-a9e5-22c05d7b8bb5 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:44:45.092704 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b398ad-54b3-4eb8-a9e5-22c05d7b8bb5 None None] PATH_INFO: `/` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:44:45.097417 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1316/2628] 10.222.0.33 () {54 vars in 921 bytes} [Mon Jul 25 05:44:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:44:45.101459 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24fb585-3954-43d6-b1fc-d1b3d8869e72 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:44:45.101974 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24fb585-3954-43d6-b1fc-d1b3d8869e72 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:44:45.102090 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24fb585-3954-43d6-b1fc-d1b3d8869e72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:44:45.135684 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-e24fb585-3954-43d6-b1fc-d1b3d8869e72 None None] MFA Rules not processed for user `1cd482a61e8a420db45bb532e5dae6e2`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:44:45.141025 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-e24fb585-3954-43d6-b1fc-d1b3d8869e72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:44:45.218058 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1313/2629] 10.222.0.33 () {58 vars in 1018 bytes} [Mon Jul 25 05:44:45 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:44:45.230370 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b398ad-54b3-4eb8-a9e5-22c05d7b8bb5 None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:44:45.232144 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-d6b398ad-54b3-4eb8-a9e5-22c05d7b8bb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:44:45.309037 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b398ad-54b3-4eb8-a9e5-22c05d7b8bb5 None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:44:45.313608 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10bcebba-f172-4785-b0b2-b9a41e193c0c service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de99e44bdf9c4030adbe9c0efa3831cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92afba266cce4d1bad4134c351e74be3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:44:45.314293 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10bcebba-f172-4785-b0b2-b9a41e193c0c service glance] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:44:45.314583 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10bcebba-f172-4785-b0b2-b9a41e193c0c service glance] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:44:45.314883 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10bcebba-f172-4785-b0b2-b9a41e193c0c service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:44:45.327301 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10bcebba-f172-4785-b0b2-b9a41e193c0c service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:44:45.328805 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10bcebba-f172-4785-b0b2-b9a41e193c0c service glance] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:44:45.340311 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1317/2630] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jul 25 05:44:45 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 05:44:45.407715 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83406f2d-0e4d-4e0e-a758-9bec5fdb6c4e None None] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:44:45.408009 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83406f2d-0e4d-4e0e-a758-9bec5fdb6c4e None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:44:45.408290 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83406f2d-0e4d-4e0e-a758-9bec5fdb6c4e None None] PATH_INFO: `/` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:44:45.409711 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1314/2631] 10.222.0.33 () {54 vars in 921 bytes} [Mon Jul 25 05:44:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:44:45.414348 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca20fb1-2a45-4754-9962-823767260d75 None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:44:45.414673 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca20fb1-2a45-4754-9962-823767260d75 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:44:45.415021 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca20fb1-2a45-4754-9962-823767260d75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:44:45.436682 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-6ca20fb1-2a45-4754-9962-823767260d75 None None] MFA Rules not processed for user `b2931200f99e43d79a50a9f6e02738e6`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:44:45.438150 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-6ca20fb1-2a45-4754-9962-823767260d75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:44:45.461401 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:44:45.492677 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1318/2632] 10.222.0.33 () {58 vars in 1018 bytes} [Mon Jul 25 05:44:45 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:44:45.505861 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83406f2d-0e4d-4e0e-a758-9bec5fdb6c4e None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:44:45.507554 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-83406f2d-0e4d-4e0e-a758-9bec5fdb6c4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:44:45.521581 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83406f2d-0e4d-4e0e-a758-9bec5fdb6c4e None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:44:45.525224 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15e0d35-95be-4a3c-aea3-69bc2fa32b33 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b2931200f99e43d79a50a9f6e02738e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92afba266cce4d1bad4134c351e74be3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:44:45.525862 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15e0d35-95be-4a3c-aea3-69bc2fa32b33 service neutron] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:44:45.526146 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15e0d35-95be-4a3c-aea3-69bc2fa32b33 service neutron] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:44:45.526422 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15e0d35-95be-4a3c-aea3-69bc2fa32b33 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:44:45.533450 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d15e0d35-95be-4a3c-aea3-69bc2fa32b33 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:44:45.534872 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d15e0d35-95be-4a3c-aea3-69bc2fa32b33 service neutron] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:44:45.544116 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1315/2633] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jul 25 05:44:45 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 05:44:45.630996 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7488e04a-8bc0-4faf-bee8-118324d25667 None None] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:44:45.631299 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7488e04a-8bc0-4faf-bee8-118324d25667 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:44:45.631582 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7488e04a-8bc0-4faf-bee8-118324d25667 None None] PATH_INFO: `/` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:44:45.633274 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1319/2634] 10.222.0.33 () {54 vars in 921 bytes} [Mon Jul 25 05:44:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:44:45.639096 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ddaea6-0638-41a4-a9de-50d8e717433e None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:44:45.639384 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ddaea6-0638-41a4-a9de-50d8e717433e None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:44:45.639665 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ddaea6-0638-41a4-a9de-50d8e717433e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:44:45.659593 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-03ddaea6-0638-41a4-a9de-50d8e717433e None None] MFA Rules not processed for user `1cd482a61e8a420db45bb532e5dae6e2`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:44:45.661200 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-03ddaea6-0638-41a4-a9de-50d8e717433e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:44:45.672065 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1316/2635] 10.222.0.33 () {58 vars in 1018 bytes} [Mon Jul 25 05:44:45 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:44:45.682580 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7488e04a-8bc0-4faf-bee8-118324d25667 None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:44:45.695297 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7488e04a-8bc0-4faf-bee8-118324d25667 None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:44:45.699833 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d07b3910-31aa-4090-9e09-ac16b31fb76b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b2931200f99e43d79a50a9f6e02738e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92afba266cce4d1bad4134c351e74be3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:44:45.700487 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07b3910-31aa-4090-9e09-ac16b31fb76b service neutron] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:44:45.700805 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07b3910-31aa-4090-9e09-ac16b31fb76b service neutron] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:44:45.701087 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07b3910-31aa-4090-9e09-ac16b31fb76b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:44:45.712269 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d07b3910-31aa-4090-9e09-ac16b31fb76b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:44:45.713759 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d07b3910-31aa-4090-9e09-ac16b31fb76b service neutron] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:44:45.725227 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1320/2636] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jul 25 05:44:45 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 25 05:44:47.818793 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ddaea6-0638-41a4-a9de-50d8e717433e None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:44:47.828241 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:44:47.900584 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ddaea6-0638-41a4-a9de-50d8e717433e None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:44:47.907480 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-067798f2-3a66-48ce-81a8-195e4677f168 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:44:47.908810 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067798f2-3a66-48ce-81a8-195e4677f168 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:44:47.909038 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067798f2-3a66-48ce-81a8-195e4677f168 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:44:47.911294 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067798f2-3a66-48ce-81a8-195e4677f168 admin admin] PATH_INFO: `/v3/users/217c3c2b10d14adb8e94a2840cde8188` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:44:47.912067 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-067798f2-3a66-48ce-81a8-195e4677f168 admin admin] RBAC: Authorizing `identity:delete_user(user_id=217c3c2b10d14adb8e94a2840cde8188)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:44:47.914144 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-067798f2-3a66-48ce-81a8-195e4677f168 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:44:47.976407 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-067798f2-3a66-48ce-81a8-195e4677f168 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '217c3c2b10d14adb8e94a2840cde8188'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:44:47.976679 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-067798f2-3a66-48ce-81a8-195e4677f168 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '217c3c2b10d14adb8e94a2840cde8188'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:44:47.985412 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-067798f2-3a66-48ce-81a8-195e4677f168 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '217c3c2b10d14adb8e94a2840cde8188'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:44:48.002513 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-067798f2-3a66-48ce-81a8-195e4677f168 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '217c3c2b10d14adb8e94a2840cde8188'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:44:48.003251 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-067798f2-3a66-48ce-81a8-195e4677f168 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '217c3c2b10d14adb8e94a2840cde8188'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:44:48.021874 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1317/2637] 10.222.0.33 () {58 vars in 1223 bytes} [Mon Jul 25 05:44:47 2022] DELETE /identity/v3/users/217c3c2b10d14adb8e94a2840cde8188 => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 05:44:48.027012 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b903ac71-50cf-424d-92f5-87ab4c71714e None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:44:48.027525 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b903ac71-50cf-424d-92f5-87ab4c71714e None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:44:48.028308 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b903ac71-50cf-424d-92f5-87ab4c71714e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:44:48.046726 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-b903ac71-50cf-424d-92f5-87ab4c71714e None None] MFA Rules not processed for user `1eae540242d94c10b0ad33960a855f08`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:44:48.048393 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-b903ac71-50cf-424d-92f5-87ab4c71714e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:44:48.127311 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1321/2638] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 25 05:44:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:44:48.141862 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-067798f2-3a66-48ce-81a8-195e4677f168 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:44:48.143554 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-067798f2-3a66-48ce-81a8-195e4677f168 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:44:48.275339 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-067798f2-3a66-48ce-81a8-195e4677f168 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:44:48.281288 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-012e9b5d-45b1-4546-a21e-7547ca685ec2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b2931200f99e43d79a50a9f6e02738e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92afba266cce4d1bad4134c351e74be3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:44:48.282043 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012e9b5d-45b1-4546-a21e-7547ca685ec2 service neutron] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:44:48.282334 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012e9b5d-45b1-4546-a21e-7547ca685ec2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:44:48.282616 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012e9b5d-45b1-4546-a21e-7547ca685ec2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:44:48.316682 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-012e9b5d-45b1-4546-a21e-7547ca685ec2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:44:48.319613 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-012e9b5d-45b1-4546-a21e-7547ca685ec2 service neutron] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:44:48.341146 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1318/2639] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jul 25 05:44:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 200 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 05:44:48.773616 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b903ac71-50cf-424d-92f5-87ab4c71714e None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:44:48.775966 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-b903ac71-50cf-424d-92f5-87ab4c71714e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:44:48.816574 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b903ac71-50cf-424d-92f5-87ab4c71714e None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:44:48.831831 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eedef2e0-8447-4c9c-820b-fb069eaf438b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:44:48.832718 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eedef2e0-8447-4c9c-820b-fb069eaf438b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:44:48.833030 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eedef2e0-8447-4c9c-820b-fb069eaf438b admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:44:48.833306 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eedef2e0-8447-4c9c-820b-fb069eaf438b admin admin] PATH_INFO: `/v3/projects/04ff486cc2c14efc954afcb2465c8194` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:44:48.840036 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eedef2e0-8447-4c9c-820b-fb069eaf438b admin admin] RBAC: Authorizing `identity:delete_project(project_id=04ff486cc2c14efc954afcb2465c8194)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:44:48.842058 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eedef2e0-8447-4c9c-820b-fb069eaf438b admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:44:48.939560 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-eedef2e0-8447-4c9c-820b-fb069eaf438b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '04ff486cc2c14efc954afcb2465c8194', 'request_id': 'req-eedef2e0-8447-4c9c-820b-fb069eaf438b'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:44:48.949909 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-eedef2e0-8447-4c9c-820b-fb069eaf438b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '04ff486cc2c14efc954afcb2465c8194', 'request_id': 'req-eedef2e0-8447-4c9c-820b-fb069eaf438b'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:44:48.958556 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-eedef2e0-8447-4c9c-820b-fb069eaf438b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '04ff486cc2c14efc954afcb2465c8194', 'request_id': 'req-eedef2e0-8447-4c9c-820b-fb069eaf438b'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:44:48.978034 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-eedef2e0-8447-4c9c-820b-fb069eaf438b admin admin] The token cache is being invalidate because project 04ff486cc2c14efc954afcb2465c8194 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80248) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 05:44:48.979133 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-eedef2e0-8447-4c9c-820b-fb069eaf438b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:44:48.979801 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-eedef2e0-8447-4c9c-820b-fb069eaf438b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:44:48.984505 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1322/2640] 10.222.0.33 () {58 vars in 1232 bytes} [Mon Jul 25 05:44:48 2022] DELETE /identity/v3/projects/04ff486cc2c14efc954afcb2465c8194 => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 05:44:48.985980 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-012e9b5d-45b1-4546-a21e-7547ca685ec2 service neutron] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:44:48.997573 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-012e9b5d-45b1-4546-a21e-7547ca685ec2 service neutron] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:44:49.032068 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:44:49.061373 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c81f8d7-1483-4c70-9eef-2b4578561336 None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:44:49.062217 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c81f8d7-1483-4c70-9eef-2b4578561336 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:44:49.062815 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c81f8d7-1483-4c70-9eef-2b4578561336 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:44:49.076150 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a2f3a6-7409-4058-a796-7ea3e88d6bad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:44:49.077233 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a2f3a6-7409-4058-a796-7ea3e88d6bad admin admin] REQUEST_METHOD: `DELETE` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:44:49.077621 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a2f3a6-7409-4058-a796-7ea3e88d6bad admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:44:49.078000 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a2f3a6-7409-4058-a796-7ea3e88d6bad admin admin] PATH_INFO: `/v3/users/100356cea3244b0591d174892ba0d132` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:44:49.079478 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8a2f3a6-7409-4058-a796-7ea3e88d6bad admin admin] RBAC: Authorizing `identity:delete_user(user_id=100356cea3244b0591d174892ba0d132)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:44:49.082525 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8a2f3a6-7409-4058-a796-7ea3e88d6bad admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:44:49.088192 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-6c81f8d7-1483-4c70-9eef-2b4578561336 None None] MFA Rules not processed for user `3352531be1b54f76b864383ccc2e5ec2`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:44:49.089481 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-6c81f8d7-1483-4c70-9eef-2b4578561336 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:44:49.142524 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:44:49.148195 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-f8a2f3a6-7409-4058-a796-7ea3e88d6bad admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '100356cea3244b0591d174892ba0d132'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:44:49.149008 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-f8a2f3a6-7409-4058-a796-7ea3e88d6bad admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '100356cea3244b0591d174892ba0d132'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:44:49.158887 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-f8a2f3a6-7409-4058-a796-7ea3e88d6bad admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '100356cea3244b0591d174892ba0d132'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:44:49.167330 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1323/2641] 10.222.0.33 () {58 vars in 1009 bytes} [Mon Jul 25 05:44:49 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 05:44:49.174982 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-f8a2f3a6-7409-4058-a796-7ea3e88d6bad admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '100356cea3244b0591d174892ba0d132'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:44:49.175811 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-f8a2f3a6-7409-4058-a796-7ea3e88d6bad admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '100356cea3244b0591d174892ba0d132'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:44:49.177217 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c81f8d7-1483-4c70-9eef-2b4578561336 None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:44:49.179369 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-6c81f8d7-1483-4c70-9eef-2b4578561336 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:44:49.200877 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1319/2642] 10.222.0.33 () {58 vars in 1223 bytes} [Mon Jul 25 05:44:48 2022] DELETE /identity/v3/users/100356cea3244b0591d174892ba0d132 => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 25 05:44:49.268562 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c81f8d7-1483-4c70-9eef-2b4578561336 None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:44:49.348891 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-039f98ad-8632-46e5-9c48-d8747cac69aa service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb135a8d5d7444cfbfa336115ba2ba44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92afba266cce4d1bad4134c351e74be3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:44:49.350122 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039f98ad-8632-46e5-9c48-d8747cac69aa service swift] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:44:49.350508 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039f98ad-8632-46e5-9c48-d8747cac69aa service swift] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:44:49.350886 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039f98ad-8632-46e5-9c48-d8747cac69aa service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:44:49.354268 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-039f98ad-8632-46e5-9c48-d8747cac69aa service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:44:49.433193 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-039f98ad-8632-46e5-9c48-d8747cac69aa service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:44:49.434792 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-039f98ad-8632-46e5-9c48-d8747cac69aa service swift] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:44:49.445801 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1324/2643] 10.222.0.33 () {60 vars in 1384 bytes} [Mon Jul 25 05:44:49 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 269 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jul 25 05:44:49.739886 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a2f3a6-7409-4058-a796-7ea3e88d6bad admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:44:49.750724 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-f8a2f3a6-7409-4058-a796-7ea3e88d6bad admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:44:49.870406 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a2f3a6-7409-4058-a796-7ea3e88d6bad admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:44:49.877462 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d97fc06-d7e9-40ef-8ea2-eee0fb9fbba5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:44:49.888815 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d97fc06-d7e9-40ef-8ea2-eee0fb9fbba5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:44:49.888815 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d97fc06-d7e9-40ef-8ea2-eee0fb9fbba5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:44:49.888815 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d97fc06-d7e9-40ef-8ea2-eee0fb9fbba5 admin admin] PATH_INFO: `/v3/projects/0f6d934558b141c7880c0523cb9767c5` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:44:49.888815 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d97fc06-d7e9-40ef-8ea2-eee0fb9fbba5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0f6d934558b141c7880c0523cb9767c5)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:44:49.888815 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d97fc06-d7e9-40ef-8ea2-eee0fb9fbba5 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:44:49.994581 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:44:49.996435 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-3d97fc06-d7e9-40ef-8ea2-eee0fb9fbba5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0f6d934558b141c7880c0523cb9767c5', 'request_id': 'req-3d97fc06-d7e9-40ef-8ea2-eee0fb9fbba5'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:44:50.007874 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-3d97fc06-d7e9-40ef-8ea2-eee0fb9fbba5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0f6d934558b141c7880c0523cb9767c5', 'request_id': 'req-3d97fc06-d7e9-40ef-8ea2-eee0fb9fbba5'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:44:50.032168 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-3d97fc06-d7e9-40ef-8ea2-eee0fb9fbba5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0f6d934558b141c7880c0523cb9767c5', 'request_id': 'req-3d97fc06-d7e9-40ef-8ea2-eee0fb9fbba5'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:44:50.056956 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-3d97fc06-d7e9-40ef-8ea2-eee0fb9fbba5 admin admin] The token cache is being invalidate because project 0f6d934558b141c7880c0523cb9767c5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80249) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 05:44:50.057408 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-3d97fc06-d7e9-40ef-8ea2-eee0fb9fbba5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:44:50.058143 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-3d97fc06-d7e9-40ef-8ea2-eee0fb9fbba5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:44:50.060345 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1320/2644] 10.222.0.33 () {58 vars in 1232 bytes} [Mon Jul 25 05:44:49 2022] DELETE /identity/v3/projects/0f6d934558b141c7880c0523cb9767c5 => generated 0 bytes in 321 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 05:44:50.065185 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7325149-dd54-485e-a5f5-1d827b74fa34 None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:44:50.066577 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7325149-dd54-485e-a5f5-1d827b74fa34 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:44:50.067063 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7325149-dd54-485e-a5f5-1d827b74fa34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:44:50.095363 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-f7325149-dd54-485e-a5f5-1d827b74fa34 None None] MFA Rules not processed for user `1eae540242d94c10b0ad33960a855f08`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:44:50.097138 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-f7325149-dd54-485e-a5f5-1d827b74fa34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:44:50.170813 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:44:50.240057 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1325/2645] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 25 05:44:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:44:50.241824 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb023846-efbf-4b19-8416-29f08cf7bcf6 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:44:50.242118 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb023846-efbf-4b19-8416-29f08cf7bcf6 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:44:50.242407 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb023846-efbf-4b19-8416-29f08cf7bcf6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:44:50.280057 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-eb023846-efbf-4b19-8416-29f08cf7bcf6 None None] MFA Rules not processed for user `1eae540242d94c10b0ad33960a855f08`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:44:50.281354 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-eb023846-efbf-4b19-8416-29f08cf7bcf6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:44:50.310132 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1321/2646] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 25 05:44:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 66 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:44:50.312113 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7325149-dd54-485e-a5f5-1d827b74fa34 None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:44:50.342090 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7325149-dd54-485e-a5f5-1d827b74fa34 None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:44:50.367302 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61894df2-8a80-47c3-a05d-08e9ff0b1ca7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:44:50.368966 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61894df2-8a80-47c3-a05d-08e9ff0b1ca7 admin admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:44:50.368966 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61894df2-8a80-47c3-a05d-08e9ff0b1ca7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:44:50.369199 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61894df2-8a80-47c3-a05d-08e9ff0b1ca7 admin admin] PATH_INFO: `/v3/domains` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:44:50.369961 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61894df2-8a80-47c3-a05d-08e9ff0b1ca7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:44:50.370274 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61894df2-8a80-47c3-a05d-08e9ff0b1ca7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:44:50.381757 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61894df2-8a80-47c3-a05d-08e9ff0b1ca7 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:44:50.409256 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1326/2647] 10.222.0.33 () {58 vars in 1152 bytes} [Mon Jul 25 05:44:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:44:50.420544 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb023846-efbf-4b19-8416-29f08cf7bcf6 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:44:50.430507 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb023846-efbf-4b19-8416-29f08cf7bcf6 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:44:50.452765 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-262218b6-75ff-4c53-89cf-5f29764c988e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:44:50.453465 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262218b6-75ff-4c53-89cf-5f29764c988e admin admin] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:44:50.453753 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262218b6-75ff-4c53-89cf-5f29764c988e admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:44:50.454031 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262218b6-75ff-4c53-89cf-5f29764c988e admin admin] PATH_INFO: `/v3/projects` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:44:50.454762 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-262218b6-75ff-4c53-89cf-5f29764c988e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:44:50.464139 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-262218b6-75ff-4c53-89cf-5f29764c988e admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:44:50.528005 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1322/2648] 10.222.0.33 () {60 vars in 1152 bytes} [Mon Jul 25 05:44:50 2022] POST /identity/v3/projects => generated 393 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 05:44:50.531782 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61894df2-8a80-47c3-a05d-08e9ff0b1ca7 admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:44:50.544977 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61894df2-8a80-47c3-a05d-08e9ff0b1ca7 admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:44:50.646236 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c78b675e-b329-4c33-9a3f-3b5c609e09d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:44:50.647298 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78b675e-b329-4c33-9a3f-3b5c609e09d3 admin admin] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:44:50.648653 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78b675e-b329-4c33-9a3f-3b5c609e09d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:44:50.648653 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78b675e-b329-4c33-9a3f-3b5c609e09d3 admin admin] PATH_INFO: `/v3/users` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:44:50.650407 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c78b675e-b329-4c33-9a3f-3b5c609e09d3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:44:50.654450 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c78b675e-b329-4c33-9a3f-3b5c609e09d3 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:44:50.692832 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1327/2649] 10.222.0.33 () {60 vars in 1143 bytes} [Mon Jul 25 05:44:50 2022] POST /identity/v3/users => generated 297 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 05:44:50.695712 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-262218b6-75ff-4c53-89cf-5f29764c988e admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:44:50.712445 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-262218b6-75ff-4c53-89cf-5f29764c988e admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:44:50.738859 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e54241-8956-4b13-a4ce-b32c537e1289 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:44:50.740135 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e54241-8956-4b13-a4ce-b32c537e1289 admin admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:44:50.740599 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e54241-8956-4b13-a4ce-b32c537e1289 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:44:50.741041 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e54241-8956-4b13-a4ce-b32c537e1289 admin admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:44:50.749103 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1e54241-8956-4b13-a4ce-b32c537e1289 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:44:50.755546 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1e54241-8956-4b13-a4ce-b32c537e1289 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:44:50.790623 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1323/2650] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jul 25 05:44:50 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 05:44:50.791722 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c78b675e-b329-4c33-9a3f-3b5c609e09d3 admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:44:50.836488 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c78b675e-b329-4c33-9a3f-3b5c609e09d3 admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:44:50.857202 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03cdf76a-018d-41e0-b1aa-0dfae3eb8bcb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:44:50.858304 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03cdf76a-018d-41e0-b1aa-0dfae3eb8bcb admin admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:44:50.860329 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03cdf76a-018d-41e0-b1aa-0dfae3eb8bcb admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:44:50.860896 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03cdf76a-018d-41e0-b1aa-0dfae3eb8bcb admin admin] PATH_INFO: `/v3/roles` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:44:50.866671 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03cdf76a-018d-41e0-b1aa-0dfae3eb8bcb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:44:50.875949 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03cdf76a-018d-41e0-b1aa-0dfae3eb8bcb admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:44:50.894876 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e54241-8956-4b13-a4ce-b32c537e1289 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:44:50.895554 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1328/2651] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jul 25 05:44:50 2022] GET /identity/v3/roles => generated 1420 bytes in 103 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 25 05:44:50.911713 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e54241-8956-4b13-a4ce-b32c537e1289 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:44:50.922796 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42140fd7-4a74-4687-94e8-3d5d48419ac9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:44:50.924849 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42140fd7-4a74-4687-94e8-3d5d48419ac9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:44:50.925352 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42140fd7-4a74-4687-94e8-3d5d48419ac9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:44:50.926461 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42140fd7-4a74-4687-94e8-3d5d48419ac9 admin admin] PATH_INFO: `/v3/projects/459689b728e14e7498c0c3ede03d95e1/users/456655a369d24818abea9edff61db71a/roles/5074e74e35f54d4b9bb415c34999a82a` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:44:50.949862 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42140fd7-4a74-4687-94e8-3d5d48419ac9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=459689b728e14e7498c0c3ede03d95e1, user_id=456655a369d24818abea9edff61db71a, role_id=5074e74e35f54d4b9bb415c34999a82a)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:44:50.954258 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42140fd7-4a74-4687-94e8-3d5d48419ac9 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:44:50.982103 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1324/2652] 10.222.0.33 () {60 vars in 1482 bytes} [Mon Jul 25 05:44:50 2022] PUT /identity/v3/projects/459689b728e14e7498c0c3ede03d95e1/users/456655a369d24818abea9edff61db71a/roles/5074e74e35f54d4b9bb415c34999a82a => generated 0 bytes in 85 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 05:44:50.993424 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86cb5cb-307d-48a9-81cc-15ef0afcadda None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:44:50.993875 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86cb5cb-307d-48a9-81cc-15ef0afcadda None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:44:50.994282 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86cb5cb-307d-48a9-81cc-15ef0afcadda None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:44:51.031764 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-b86cb5cb-307d-48a9-81cc-15ef0afcadda None None] MFA Rules not processed for user `456655a369d24818abea9edff61db71a`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:44:51.033589 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-b86cb5cb-307d-48a9-81cc-15ef0afcadda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:44:51.173088 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1329/2653] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 25 05:44:50 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:44:51.178247 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42140fd7-4a74-4687-94e8-3d5d48419ac9 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:44:51.276767 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42140fd7-4a74-4687-94e8-3d5d48419ac9 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:44:51.283109 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-506093dc-5c67-4fad-9d55-3687273db3ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:44:51.284378 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506093dc-5c67-4fad-9d55-3687273db3ca admin admin] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:44:51.284960 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506093dc-5c67-4fad-9d55-3687273db3ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:44:51.285440 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506093dc-5c67-4fad-9d55-3687273db3ca admin admin] PATH_INFO: `/v3/projects` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:44:51.286856 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-506093dc-5c67-4fad-9d55-3687273db3ca admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:44:51.290761 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-506093dc-5c67-4fad-9d55-3687273db3ca admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:44:51.355347 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1325/2654] 10.222.0.33 () {60 vars in 1152 bytes} [Mon Jul 25 05:44:51 2022] POST /identity/v3/projects => generated 393 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 05:44:51.356633 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b86cb5cb-307d-48a9-81cc-15ef0afcadda None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:44:51.375013 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b86cb5cb-307d-48a9-81cc-15ef0afcadda None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:44:51.456716 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:44:51.514503 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f265167-0069-48f7-aff3-60e1f9cf8672 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:44:51.515404 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f265167-0069-48f7-aff3-60e1f9cf8672 admin admin] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:44:51.515705 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f265167-0069-48f7-aff3-60e1f9cf8672 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:44:51.516062 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f265167-0069-48f7-aff3-60e1f9cf8672 admin admin] PATH_INFO: `/v3/users` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:44:51.516946 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f265167-0069-48f7-aff3-60e1f9cf8672 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:44:51.527130 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f265167-0069-48f7-aff3-60e1f9cf8672 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:44:51.576818 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1330/2655] 10.222.0.33 () {60 vars in 1143 bytes} [Mon Jul 25 05:44:51 2022] POST /identity/v3/users => generated 303 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 05:44:51.579361 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-506093dc-5c67-4fad-9d55-3687273db3ca admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:44:51.602383 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-506093dc-5c67-4fad-9d55-3687273db3ca admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:44:51.628520 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42aad141-7955-49db-9dd0-ef6427ce400d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:44:51.629551 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42aad141-7955-49db-9dd0-ef6427ce400d admin admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:44:51.630595 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42aad141-7955-49db-9dd0-ef6427ce400d admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:44:51.631059 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42aad141-7955-49db-9dd0-ef6427ce400d admin admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:44:51.632878 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42aad141-7955-49db-9dd0-ef6427ce400d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:44:51.637175 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42aad141-7955-49db-9dd0-ef6427ce400d admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:44:51.659879 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1326/2656] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jul 25 05:44:51 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 05:44:51.660894 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f265167-0069-48f7-aff3-60e1f9cf8672 admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:44:51.674775 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f265167-0069-48f7-aff3-60e1f9cf8672 admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:44:51.685535 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc2e038-9c66-4907-b49c-e5724b0c481d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:44:51.686558 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc2e038-9c66-4907-b49c-e5724b0c481d admin admin] REQUEST_METHOD: `PUT` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:44:51.686920 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc2e038-9c66-4907-b49c-e5724b0c481d admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:44:51.687316 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc2e038-9c66-4907-b49c-e5724b0c481d admin admin] PATH_INFO: `/v3/projects/e9bb5abd52ea4c7b85067435ac74016f/users/c261f5443f3946ed9569eddda0c1a330/roles/39e5c666285a42fe8fc2c3afca6260d1` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:44:51.728225 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fc2e038-9c66-4907-b49c-e5724b0c481d admin admin] RBAC: Authorizing `identity:create_grant(project_id=e9bb5abd52ea4c7b85067435ac74016f, user_id=c261f5443f3946ed9569eddda0c1a330, role_id=39e5c666285a42fe8fc2c3afca6260d1)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:44:51.731168 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fc2e038-9c66-4907-b49c-e5724b0c481d admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:44:51.755821 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1331/2657] 10.222.0.33 () {60 vars in 1482 bytes} [Mon Jul 25 05:44:51 2022] PUT /identity/v3/projects/e9bb5abd52ea4c7b85067435ac74016f/users/c261f5443f3946ed9569eddda0c1a330/roles/39e5c666285a42fe8fc2c3afca6260d1 => generated 0 bytes in 96 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 25 05:44:51.761056 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42aad141-7955-49db-9dd0-ef6427ce400d admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:44:51.778180 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42aad141-7955-49db-9dd0-ef6427ce400d admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:44:51.851035 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=93,keystone:DELETE=3,keystone:INSERT=4 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:44:51.869271 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57199b9b-0f34-4d63-9b9f-f7b183cda661 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:44:51.869935 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57199b9b-0f34-4d63-9b9f-f7b183cda661 admin admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:44:51.870233 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57199b9b-0f34-4d63-9b9f-f7b183cda661 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:44:51.870515 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57199b9b-0f34-4d63-9b9f-f7b183cda661 admin admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:44:51.871482 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57199b9b-0f34-4d63-9b9f-f7b183cda661 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:44:51.873694 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57199b9b-0f34-4d63-9b9f-f7b183cda661 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:44:51.893789 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1327/2658] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jul 25 05:44:51 2022] GET /identity/v3/roles => generated 1420 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 05:44:51.897944 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc2e038-9c66-4907-b49c-e5724b0c481d admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:44:51.910766 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc2e038-9c66-4907-b49c-e5724b0c481d admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:44:51.916974 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b1bbd8-2f15-44bf-9f18-541c83924467 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:44:51.917981 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b1bbd8-2f15-44bf-9f18-541c83924467 admin admin] REQUEST_METHOD: `PUT` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:44:51.918345 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b1bbd8-2f15-44bf-9f18-541c83924467 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:44:51.918699 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b1bbd8-2f15-44bf-9f18-541c83924467 admin admin] PATH_INFO: `/v3/projects/e9bb5abd52ea4c7b85067435ac74016f/users/c261f5443f3946ed9569eddda0c1a330/roles/39e5c666285a42fe8fc2c3afca6260d1` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:44:51.920907 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4b1bbd8-2f15-44bf-9f18-541c83924467 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e9bb5abd52ea4c7b85067435ac74016f, user_id=c261f5443f3946ed9569eddda0c1a330, role_id=39e5c666285a42fe8fc2c3afca6260d1)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:44:51.924297 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4b1bbd8-2f15-44bf-9f18-541c83924467 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:44:51.956207 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1332/2659] 10.222.0.33 () {60 vars in 1482 bytes} [Mon Jul 25 05:44:51 2022] PUT /identity/v3/projects/e9bb5abd52ea4c7b85067435ac74016f/users/c261f5443f3946ed9569eddda0c1a330/roles/39e5c666285a42fe8fc2c3afca6260d1 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 25 05:44:51.958391 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855b3ed4-f70f-4dac-8508-477c288cfd2a None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:44:51.958880 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855b3ed4-f70f-4dac-8508-477c288cfd2a None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:44:51.959220 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855b3ed4-f70f-4dac-8508-477c288cfd2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:44:51.999338 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-855b3ed4-f70f-4dac-8508-477c288cfd2a None None] MFA Rules not processed for user `c261f5443f3946ed9569eddda0c1a330`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:44:52.000731 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-855b3ed4-f70f-4dac-8508-477c288cfd2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:44:52.090705 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1328/2660] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 25 05:44:51 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:44:52.103692 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b1bbd8-2f15-44bf-9f18-541c83924467 admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:44:52.105754 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-d4b1bbd8-2f15-44bf-9f18-541c83924467 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:44:52.217135 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b1bbd8-2f15-44bf-9f18-541c83924467 admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:44:52.221002 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-765dca28-75e0-4636-a8da-7a4594c066d6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db57b478c25643dbaf936f90f6e17d0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92afba266cce4d1bad4134c351e74be3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:44:52.221909 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765dca28-75e0-4636-a8da-7a4594c066d6 service cinder] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:44:52.222305 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765dca28-75e0-4636-a8da-7a4594c066d6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:44:52.222649 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765dca28-75e0-4636-a8da-7a4594c066d6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:44:52.240016 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-765dca28-75e0-4636-a8da-7a4594c066d6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:44:52.240016 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-765dca28-75e0-4636-a8da-7a4594c066d6 service cinder] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:44:52.246936 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1333/2661] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jul 25 05:44:52 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 05:44:52.298886 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-855b3ed4-f70f-4dac-8508-477c288cfd2a None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:44:52.300392 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-855b3ed4-f70f-4dac-8508-477c288cfd2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:44:52.313600 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-855b3ed4-f70f-4dac-8508-477c288cfd2a None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:44:52.318078 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-238b0f4f-793c-4908-a7ee-7185a299e761 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db57b478c25643dbaf936f90f6e17d0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92afba266cce4d1bad4134c351e74be3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:44:52.318773 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238b0f4f-793c-4908-a7ee-7185a299e761 service cinder] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:44:52.319064 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238b0f4f-793c-4908-a7ee-7185a299e761 service cinder] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:44:52.319345 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238b0f4f-793c-4908-a7ee-7185a299e761 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:44:52.389109 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-238b0f4f-793c-4908-a7ee-7185a299e761 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:44:52.390535 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-238b0f4f-793c-4908-a7ee-7185a299e761 service cinder] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:44:52.401283 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1329/2662] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jul 25 05:44:52 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 05:44:55.831426 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-765dca28-75e0-4636-a8da-7a4594c066d6 service cinder] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:44:55.833041 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-765dca28-75e0-4636-a8da-7a4594c066d6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:44:55.905834 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-765dca28-75e0-4636-a8da-7a4594c066d6 service cinder] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:44:55.911882 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c9c8b74-a7f3-4924-ae54-710bc39d6952 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:44:55.912790 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9c8b74-a7f3-4924-ae54-710bc39d6952 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:44:55.913090 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9c8b74-a7f3-4924-ae54-710bc39d6952 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:44:55.913369 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9c8b74-a7f3-4924-ae54-710bc39d6952 admin admin] PATH_INFO: `/v3/users/ad83ed1e819346f3ab0de83c8cb83278` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:44:55.914575 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c9c8b74-a7f3-4924-ae54-710bc39d6952 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ad83ed1e819346f3ab0de83c8cb83278)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:44:55.916981 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c9c8b74-a7f3-4924-ae54-710bc39d6952 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:44:55.960800 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=90,keystone:INSERT=5,keystone:DELETE=5 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:44:55.985400 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-5c9c8b74-a7f3-4924-ae54-710bc39d6952 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ad83ed1e819346f3ab0de83c8cb83278'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:44:56.010576 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-5c9c8b74-a7f3-4924-ae54-710bc39d6952 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ad83ed1e819346f3ab0de83c8cb83278'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:44:56.011430 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-5c9c8b74-a7f3-4924-ae54-710bc39d6952 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ad83ed1e819346f3ab0de83c8cb83278'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:44:56.025271 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-5c9c8b74-a7f3-4924-ae54-710bc39d6952 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ad83ed1e819346f3ab0de83c8cb83278'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:44:56.026167 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-5c9c8b74-a7f3-4924-ae54-710bc39d6952 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ad83ed1e819346f3ab0de83c8cb83278'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:44:56.046389 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1334/2663] 10.222.0.33 () {58 vars in 1223 bytes} [Mon Jul 25 05:44:55 2022] DELETE /identity/v3/users/ad83ed1e819346f3ab0de83c8cb83278 => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 05:44:56.047576 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60def5d-2551-4d2e-9cfe-6e46e225d038 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:44:56.048088 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60def5d-2551-4d2e-9cfe-6e46e225d038 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:44:56.048197 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60def5d-2551-4d2e-9cfe-6e46e225d038 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:44:56.074702 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-a60def5d-2551-4d2e-9cfe-6e46e225d038 None None] MFA Rules not processed for user `1eae540242d94c10b0ad33960a855f08`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:44:56.077392 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-a60def5d-2551-4d2e-9cfe-6e46e225d038 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:44:56.173835 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1330/2664] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 25 05:44:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:44:56.184658 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c9c8b74-a7f3-4924-ae54-710bc39d6952 admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:44:56.186327 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-5c9c8b74-a7f3-4924-ae54-710bc39d6952 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:44:56.280497 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a60def5d-2551-4d2e-9cfe-6e46e225d038 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:44:56.283764 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-a60def5d-2551-4d2e-9cfe-6e46e225d038 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:44:56.288199 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c9c8b74-a7f3-4924-ae54-710bc39d6952 admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:44:56.294304 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc6cb60b-d459-4332-adec-54269751d44a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b2931200f99e43d79a50a9f6e02738e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92afba266cce4d1bad4134c351e74be3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:44:56.294987 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6cb60b-d459-4332-adec-54269751d44a service neutron] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:44:56.295280 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6cb60b-d459-4332-adec-54269751d44a service neutron] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:44:56.295562 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6cb60b-d459-4332-adec-54269751d44a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:44:56.298137 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a60def5d-2551-4d2e-9cfe-6e46e225d038 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:44:56.316351 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc6cb60b-d459-4332-adec-54269751d44a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:44:56.318416 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc6cb60b-d459-4332-adec-54269751d44a service neutron] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:44:56.319322 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a658e9f-bcdc-4d0b-b111-1c31c7d9f294 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:44:56.320336 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a658e9f-bcdc-4d0b-b111-1c31c7d9f294 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:44:56.320629 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a658e9f-bcdc-4d0b-b111-1c31c7d9f294 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:44:56.320936 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a658e9f-bcdc-4d0b-b111-1c31c7d9f294 admin admin] PATH_INFO: `/v3/users/456655a369d24818abea9edff61db71a` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:44:56.323133 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a658e9f-bcdc-4d0b-b111-1c31c7d9f294 admin admin] RBAC: Authorizing `identity:delete_user(user_id=456655a369d24818abea9edff61db71a)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:44:56.325111 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a658e9f-bcdc-4d0b-b111-1c31c7d9f294 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:44:56.329846 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:44:56.334097 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1335/2665] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jul 25 05:44:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 150 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 05:44:56.427799 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-9a658e9f-bcdc-4d0b-b111-1c31c7d9f294 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '456655a369d24818abea9edff61db71a'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:44:56.437204 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-9a658e9f-bcdc-4d0b-b111-1c31c7d9f294 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '456655a369d24818abea9edff61db71a'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:44:56.469680 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-9a658e9f-bcdc-4d0b-b111-1c31c7d9f294 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '456655a369d24818abea9edff61db71a'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:44:56.504311 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-9a658e9f-bcdc-4d0b-b111-1c31c7d9f294 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '456655a369d24818abea9edff61db71a'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:44:56.505452 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-9a658e9f-bcdc-4d0b-b111-1c31c7d9f294 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '456655a369d24818abea9edff61db71a'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:44:56.531129 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f57ba1b-1b19-4844-b97c-951e640de117 None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:44:56.531787 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f57ba1b-1b19-4844-b97c-951e640de117 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:44:56.532383 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f57ba1b-1b19-4844-b97c-951e640de117 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:44:56.534788 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1331/2666] 10.222.0.33 () {58 vars in 1223 bytes} [Mon Jul 25 05:44:56 2022] DELETE /identity/v3/users/456655a369d24818abea9edff61db71a => generated 0 bytes in 246 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 05:44:56.555216 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-2f57ba1b-1b19-4844-b97c-951e640de117 None None] MFA Rules not processed for user `1eae540242d94c10b0ad33960a855f08`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:44:56.557667 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-2f57ba1b-1b19-4844-b97c-951e640de117 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:44:56.635203 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1336/2667] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 25 05:44:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:44:56.656371 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a658e9f-bcdc-4d0b-b111-1c31c7d9f294 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:44:56.658025 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-9a658e9f-bcdc-4d0b-b111-1c31c7d9f294 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:44:56.764052 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a658e9f-bcdc-4d0b-b111-1c31c7d9f294 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:44:56.767988 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c41fd64b-f092-44f6-861b-b6429e63ef16 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b2931200f99e43d79a50a9f6e02738e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92afba266cce4d1bad4134c351e74be3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:44:56.768660 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41fd64b-f092-44f6-861b-b6429e63ef16 service neutron] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:44:56.768979 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41fd64b-f092-44f6-861b-b6429e63ef16 service neutron] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:44:56.769258 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41fd64b-f092-44f6-861b-b6429e63ef16 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:44:56.781705 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c41fd64b-f092-44f6-861b-b6429e63ef16 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:44:56.783384 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c41fd64b-f092-44f6-861b-b6429e63ef16 service neutron] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:44:56.794625 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1332/2668] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jul 25 05:44:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 05:44:56.815655 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f57ba1b-1b19-4844-b97c-951e640de117 None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:44:56.817316 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-2f57ba1b-1b19-4844-b97c-951e640de117 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:44:56.832762 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f57ba1b-1b19-4844-b97c-951e640de117 None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:44:56.856648 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e3a9ee-28ac-4e38-b991-f73d4a81f385 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:44:56.857544 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e3a9ee-28ac-4e38-b991-f73d4a81f385 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:44:56.857835 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e3a9ee-28ac-4e38-b991-f73d4a81f385 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:44:56.858173 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e3a9ee-28ac-4e38-b991-f73d4a81f385 admin admin] PATH_INFO: `/v3/projects/ddeae234b3b04f50ba2fbbf0b5043de0` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:44:56.870515 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7e3a9ee-28ac-4e38-b991-f73d4a81f385 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ddeae234b3b04f50ba2fbbf0b5043de0)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:44:56.873453 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7e3a9ee-28ac-4e38-b991-f73d4a81f385 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:44:56.930116 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:44:56.947534 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-e7e3a9ee-28ac-4e38-b991-f73d4a81f385 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ddeae234b3b04f50ba2fbbf0b5043de0', 'request_id': 'req-e7e3a9ee-28ac-4e38-b991-f73d4a81f385'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:44:56.955367 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-e7e3a9ee-28ac-4e38-b991-f73d4a81f385 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ddeae234b3b04f50ba2fbbf0b5043de0', 'request_id': 'req-e7e3a9ee-28ac-4e38-b991-f73d4a81f385'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:44:56.969351 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-e7e3a9ee-28ac-4e38-b991-f73d4a81f385 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ddeae234b3b04f50ba2fbbf0b5043de0', 'request_id': 'req-e7e3a9ee-28ac-4e38-b991-f73d4a81f385'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:44:56.989396 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-e7e3a9ee-28ac-4e38-b991-f73d4a81f385 admin admin] The token cache is being invalidate because project ddeae234b3b04f50ba2fbbf0b5043de0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80248) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 05:44:56.990108 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-e7e3a9ee-28ac-4e38-b991-f73d4a81f385 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:44:56.991025 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-e7e3a9ee-28ac-4e38-b991-f73d4a81f385 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:44:56.993294 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1337/2669] 10.222.0.33 () {58 vars in 1232 bytes} [Mon Jul 25 05:44:56 2022] DELETE /identity/v3/projects/ddeae234b3b04f50ba2fbbf0b5043de0 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 05:44:56.998611 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ebc8060-762d-457f-8fd0-3b9fdaa74dd6 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:44:56.998899 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ebc8060-762d-457f-8fd0-3b9fdaa74dd6 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:44:56.999182 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ebc8060-762d-457f-8fd0-3b9fdaa74dd6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:44:57.036097 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-9ebc8060-762d-457f-8fd0-3b9fdaa74dd6 None None] MFA Rules not processed for user `1eae540242d94c10b0ad33960a855f08`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:44:57.045797 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-9ebc8060-762d-457f-8fd0-3b9fdaa74dd6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:44:57.159082 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1333/2670] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 25 05:44:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:44:57.162209 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9668b2-814d-4fb7-a54a-bb5c1e3797db None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:44:57.163030 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9668b2-814d-4fb7-a54a-bb5c1e3797db None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:44:57.163532 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9668b2-814d-4fb7-a54a-bb5c1e3797db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:44:57.175333 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ebc8060-762d-457f-8fd0-3b9fdaa74dd6 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:44:57.176935 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-9ebc8060-762d-457f-8fd0-3b9fdaa74dd6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:44:57.186257 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-bd9668b2-814d-4fb7-a54a-bb5c1e3797db None None] MFA Rules not processed for user `1eae540242d94c10b0ad33960a855f08`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:44:57.188521 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-bd9668b2-814d-4fb7-a54a-bb5c1e3797db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:44:57.199940 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ebc8060-762d-457f-8fd0-3b9fdaa74dd6 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:44:57.205443 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1338/2671] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 25 05:44:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:44:57.206332 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3250c6d4-8dae-425b-8676-96e06f2e6608 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:44:57.207218 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3250c6d4-8dae-425b-8676-96e06f2e6608 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:44:57.207505 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3250c6d4-8dae-425b-8676-96e06f2e6608 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:44:57.207786 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3250c6d4-8dae-425b-8676-96e06f2e6608 admin admin] PATH_INFO: `/v3/projects/459689b728e14e7498c0c3ede03d95e1` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:44:57.209133 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3250c6d4-8dae-425b-8676-96e06f2e6608 admin admin] RBAC: Authorizing `identity:delete_project(project_id=459689b728e14e7498c0c3ede03d95e1)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:44:57.209672 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd9668b2-814d-4fb7-a54a-bb5c1e3797db None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:44:57.211039 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3250c6d4-8dae-425b-8676-96e06f2e6608 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:44:57.219755 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd9668b2-814d-4fb7-a54a-bb5c1e3797db None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:44:57.225584 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f4f649-5f15-4a70-93da-a3f36b426ed5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:44:57.226249 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f4f649-5f15-4a70-93da-a3f36b426ed5 admin admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:44:57.226419 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f4f649-5f15-4a70-93da-a3f36b426ed5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:44:57.226699 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f4f649-5f15-4a70-93da-a3f36b426ed5 admin admin] PATH_INFO: `/v3/domains` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:44:57.227432 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03f4f649-5f15-4a70-93da-a3f36b426ed5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:44:57.228275 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03f4f649-5f15-4a70-93da-a3f36b426ed5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:44:57.230074 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03f4f649-5f15-4a70-93da-a3f36b426ed5 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:44:57.246954 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:44:57.249507 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1339/2672] 10.222.0.33 () {58 vars in 1152 bytes} [Mon Jul 25 05:44:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:44:57.250411 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f4f649-5f15-4a70-93da-a3f36b426ed5 admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:44:57.262195 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f4f649-5f15-4a70-93da-a3f36b426ed5 admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:44:57.267101 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb0edbfa-e11a-40b2-b54e-9172493b5fa4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:44:57.267772 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0edbfa-e11a-40b2-b54e-9172493b5fa4 admin admin] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:44:57.268056 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0edbfa-e11a-40b2-b54e-9172493b5fa4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:44:57.268333 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0edbfa-e11a-40b2-b54e-9172493b5fa4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:44:57.269187 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb0edbfa-e11a-40b2-b54e-9172493b5fa4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:44:57.270972 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb0edbfa-e11a-40b2-b54e-9172493b5fa4 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:44:57.302616 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1340/2673] 10.222.0.33 () {60 vars in 1152 bytes} [Mon Jul 25 05:44:57 2022] POST /identity/v3/projects => generated 375 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 05:44:57.304888 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-3250c6d4-8dae-425b-8676-96e06f2e6608 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '459689b728e14e7498c0c3ede03d95e1', 'request_id': 'req-3250c6d4-8dae-425b-8676-96e06f2e6608'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:44:57.307026 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb0edbfa-e11a-40b2-b54e-9172493b5fa4 admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:44:57.311648 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-3250c6d4-8dae-425b-8676-96e06f2e6608 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '459689b728e14e7498c0c3ede03d95e1', 'request_id': 'req-3250c6d4-8dae-425b-8676-96e06f2e6608'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:44:57.325597 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-3250c6d4-8dae-425b-8676-96e06f2e6608 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '459689b728e14e7498c0c3ede03d95e1', 'request_id': 'req-3250c6d4-8dae-425b-8676-96e06f2e6608'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:44:57.350355 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-3250c6d4-8dae-425b-8676-96e06f2e6608 admin admin] The token cache is being invalidate because project 459689b728e14e7498c0c3ede03d95e1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80249) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 05:44:57.350967 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-3250c6d4-8dae-425b-8676-96e06f2e6608 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:44:57.352750 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-3250c6d4-8dae-425b-8676-96e06f2e6608 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:44:57.362391 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1334/2674] 10.222.0.33 () {58 vars in 1232 bytes} [Mon Jul 25 05:44:57 2022] DELETE /identity/v3/projects/459689b728e14e7498c0c3ede03d95e1 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 25 05:44:57.364697 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3250c6d4-8dae-425b-8676-96e06f2e6608 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:44:57.367016 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-3250c6d4-8dae-425b-8676-96e06f2e6608 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:44:57.401719 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb0edbfa-e11a-40b2-b54e-9172493b5fa4 admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:44:57.413683 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab0bc8d-f0d0-4dfc-adc0-a95912b70a1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:44:57.414441 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab0bc8d-f0d0-4dfc-adc0-a95912b70a1a admin admin] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:44:57.414793 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab0bc8d-f0d0-4dfc-adc0-a95912b70a1a admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:44:57.415191 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab0bc8d-f0d0-4dfc-adc0-a95912b70a1a admin admin] PATH_INFO: `/v3/users` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:44:57.416124 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aab0bc8d-f0d0-4dfc-adc0-a95912b70a1a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:44:57.418156 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aab0bc8d-f0d0-4dfc-adc0-a95912b70a1a admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:44:57.440070 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1341/2675] 10.222.0.33 () {60 vars in 1143 bytes} [Mon Jul 25 05:44:57 2022] POST /identity/v3/users => generated 288 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 05:44:57.444040 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab0bc8d-f0d0-4dfc-adc0-a95912b70a1a admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:44:57.445758 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-aab0bc8d-f0d0-4dfc-adc0-a95912b70a1a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:44:57.446228 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3250c6d4-8dae-425b-8676-96e06f2e6608 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:44:57.451936 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad0cb1a-bb12-4701-8f2b-dd40c4cfab37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:44:57.452941 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad0cb1a-bb12-4701-8f2b-dd40c4cfab37 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:44:57.453319 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad0cb1a-bb12-4701-8f2b-dd40c4cfab37 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:44:57.453638 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad0cb1a-bb12-4701-8f2b-dd40c4cfab37 admin admin] PATH_INFO: `/v3/users/c261f5443f3946ed9569eddda0c1a330` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:44:57.455654 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ad0cb1a-bb12-4701-8f2b-dd40c4cfab37 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c261f5443f3946ed9569eddda0c1a330)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:44:57.457053 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ad0cb1a-bb12-4701-8f2b-dd40c4cfab37 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:44:57.459458 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab0bc8d-f0d0-4dfc-adc0-a95912b70a1a admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:44:57.464378 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c9a55b-40e6-4618-8da7-a99559ca7dd3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:44:57.465086 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c9a55b-40e6-4618-8da7-a99559ca7dd3 admin admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:44:57.465371 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c9a55b-40e6-4618-8da7-a99559ca7dd3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:44:57.465650 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c9a55b-40e6-4618-8da7-a99559ca7dd3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:44:57.466383 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64c9a55b-40e6-4618-8da7-a99559ca7dd3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:44:57.467975 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64c9a55b-40e6-4618-8da7-a99559ca7dd3 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:44:57.482846 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1342/2676] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jul 25 05:44:57 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 25 05:44:57.487639 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c9a55b-40e6-4618-8da7-a99559ca7dd3 admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:44:57.500504 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c9a55b-40e6-4618-8da7-a99559ca7dd3 admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:44:57.505314 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-572cf486-6db1-4fdb-bea7-b6bcf45d2735 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:44:57.505976 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572cf486-6db1-4fdb-bea7-b6bcf45d2735 admin admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:44:57.506262 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572cf486-6db1-4fdb-bea7-b6bcf45d2735 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:44:57.506542 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572cf486-6db1-4fdb-bea7-b6bcf45d2735 admin admin] PATH_INFO: `/v3/roles` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:44:57.507269 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-572cf486-6db1-4fdb-bea7-b6bcf45d2735 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:44:57.509625 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-572cf486-6db1-4fdb-bea7-b6bcf45d2735 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:44:57.510172 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-1ad0cb1a-bb12-4701-8f2b-dd40c4cfab37 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c261f5443f3946ed9569eddda0c1a330'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:44:57.511084 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-1ad0cb1a-bb12-4701-8f2b-dd40c4cfab37 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c261f5443f3946ed9569eddda0c1a330'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:44:57.525451 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1343/2677] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jul 25 05:44:57 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 25 05:44:57.525688 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-1ad0cb1a-bb12-4701-8f2b-dd40c4cfab37 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c261f5443f3946ed9569eddda0c1a330'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:44:57.526771 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-572cf486-6db1-4fdb-bea7-b6bcf45d2735 admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:44:57.537140 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-572cf486-6db1-4fdb-bea7-b6bcf45d2735 admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:44:57.538688 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-1ad0cb1a-bb12-4701-8f2b-dd40c4cfab37 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c261f5443f3946ed9569eddda0c1a330'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:44:57.539521 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-1ad0cb1a-bb12-4701-8f2b-dd40c4cfab37 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c261f5443f3946ed9569eddda0c1a330'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:44:57.542047 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0708a89a-52b6-4e88-8110-64a13c0ebd57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:44:57.542942 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0708a89a-52b6-4e88-8110-64a13c0ebd57 admin admin] REQUEST_METHOD: `PUT` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:44:57.543237 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0708a89a-52b6-4e88-8110-64a13c0ebd57 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:44:57.543519 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0708a89a-52b6-4e88-8110-64a13c0ebd57 admin admin] PATH_INFO: `/v3/projects/4630ba0434314737a8b673a7288646f7/users/284984d3c83f443eb3e17a9647c28f9a/roles/5074e74e35f54d4b9bb415c34999a82a` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:44:57.558944 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1335/2678] 10.222.0.33 () {58 vars in 1223 bytes} [Mon Jul 25 05:44:57 2022] DELETE /identity/v3/users/c261f5443f3946ed9569eddda0c1a330 => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 05:44:57.560423 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0708a89a-52b6-4e88-8110-64a13c0ebd57 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4630ba0434314737a8b673a7288646f7, user_id=284984d3c83f443eb3e17a9647c28f9a, role_id=5074e74e35f54d4b9bb415c34999a82a)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:44:57.563096 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0708a89a-52b6-4e88-8110-64a13c0ebd57 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:44:57.584023 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1344/2679] 10.222.0.33 () {60 vars in 1482 bytes} [Mon Jul 25 05:44:57 2022] PUT /identity/v3/projects/4630ba0434314737a8b673a7288646f7/users/284984d3c83f443eb3e17a9647c28f9a/roles/5074e74e35f54d4b9bb415c34999a82a => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 25 05:44:57.590427 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3f4863-709c-4098-b2ae-41a55404d0e8 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:44:57.590846 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3f4863-709c-4098-b2ae-41a55404d0e8 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:44:57.591285 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3f4863-709c-4098-b2ae-41a55404d0e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:44:57.620552 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-be3f4863-709c-4098-b2ae-41a55404d0e8 None None] MFA Rules not processed for user `284984d3c83f443eb3e17a9647c28f9a`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:44:57.621970 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-be3f4863-709c-4098-b2ae-41a55404d0e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:44:57.636995 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:44:57.696992 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1336/2680] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 25 05:44:57 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:44:57.708477 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0708a89a-52b6-4e88-8110-64a13c0ebd57 admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:44:57.710473 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-0708a89a-52b6-4e88-8110-64a13c0ebd57 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:44:57.753352 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=90,keystone:DELETE=4,keystone:INSERT=6 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:44:57.783354 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0708a89a-52b6-4e88-8110-64a13c0ebd57 admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:44:57.792907 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28502caa-5a3a-4258-b048-cb05fa250b6a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db57b478c25643dbaf936f90f6e17d0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92afba266cce4d1bad4134c351e74be3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:44:57.793583 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28502caa-5a3a-4258-b048-cb05fa250b6a service cinder] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:44:57.793868 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28502caa-5a3a-4258-b048-cb05fa250b6a service cinder] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:44:57.794145 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28502caa-5a3a-4258-b048-cb05fa250b6a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:44:57.815085 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28502caa-5a3a-4258-b048-cb05fa250b6a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:44:57.816668 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28502caa-5a3a-4258-b048-cb05fa250b6a service cinder] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:44:57.827848 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1345/2681] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jul 25 05:44:57 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 25 05:44:58.191498 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be3f4863-709c-4098-b2ae-41a55404d0e8 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:44:58.193581 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-be3f4863-709c-4098-b2ae-41a55404d0e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:44:58.270328 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be3f4863-709c-4098-b2ae-41a55404d0e8 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:44:58.275837 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b24b296-d08b-4537-b040-bcc3017d82ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:44:58.276720 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b24b296-d08b-4537-b040-bcc3017d82ad admin admin] REQUEST_METHOD: `DELETE` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:44:58.277039 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b24b296-d08b-4537-b040-bcc3017d82ad admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:44:58.277319 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b24b296-d08b-4537-b040-bcc3017d82ad admin admin] PATH_INFO: `/v3/projects/e9bb5abd52ea4c7b85067435ac74016f` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:44:58.278548 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b24b296-d08b-4537-b040-bcc3017d82ad admin admin] RBAC: Authorizing `identity:delete_project(project_id=e9bb5abd52ea4c7b85067435ac74016f)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:44:58.280462 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b24b296-d08b-4537-b040-bcc3017d82ad admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:44:58.327039 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-8b24b296-d08b-4537-b040-bcc3017d82ad admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e9bb5abd52ea4c7b85067435ac74016f', 'request_id': 'req-8b24b296-d08b-4537-b040-bcc3017d82ad'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:44:58.332418 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-8b24b296-d08b-4537-b040-bcc3017d82ad admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e9bb5abd52ea4c7b85067435ac74016f', 'request_id': 'req-8b24b296-d08b-4537-b040-bcc3017d82ad'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:44:58.339835 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-8b24b296-d08b-4537-b040-bcc3017d82ad admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e9bb5abd52ea4c7b85067435ac74016f', 'request_id': 'req-8b24b296-d08b-4537-b040-bcc3017d82ad'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:44:58.354507 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-8b24b296-d08b-4537-b040-bcc3017d82ad admin admin] The token cache is being invalidate because project e9bb5abd52ea4c7b85067435ac74016f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80249) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 05:44:58.354791 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-8b24b296-d08b-4537-b040-bcc3017d82ad admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:44:58.355488 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-8b24b296-d08b-4537-b040-bcc3017d82ad admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:44:58.358184 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1337/2682] 10.222.0.33 () {58 vars in 1232 bytes} [Mon Jul 25 05:44:58 2022] DELETE /identity/v3/projects/e9bb5abd52ea4c7b85067435ac74016f => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 05:44:58.363361 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d605ffcf-aaaf-4f53-bfa2-e3bc1dd8c7de None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:44:58.363826 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d605ffcf-aaaf-4f53-bfa2-e3bc1dd8c7de None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:44:58.363941 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d605ffcf-aaaf-4f53-bfa2-e3bc1dd8c7de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:44:58.381756 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-d605ffcf-aaaf-4f53-bfa2-e3bc1dd8c7de None None] MFA Rules not processed for user `1eae540242d94c10b0ad33960a855f08`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:44:58.382973 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-d605ffcf-aaaf-4f53-bfa2-e3bc1dd8c7de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:44:58.442656 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1346/2683] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 25 05:44:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:44:58.450908 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe25c785-60cf-480f-a2f9-80dac486de1a None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:44:58.451202 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe25c785-60cf-480f-a2f9-80dac486de1a None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:44:58.451508 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe25c785-60cf-480f-a2f9-80dac486de1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:44:58.470268 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-fe25c785-60cf-480f-a2f9-80dac486de1a None None] MFA Rules not processed for user `1eae540242d94c10b0ad33960a855f08`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:44:58.471408 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-fe25c785-60cf-480f-a2f9-80dac486de1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:44:58.480346 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1338/2684] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 25 05:44:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:44:58.484607 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d605ffcf-aaaf-4f53-bfa2-e3bc1dd8c7de None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:44:58.494313 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d605ffcf-aaaf-4f53-bfa2-e3bc1dd8c7de None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:44:58.498830 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c33861-06fe-434f-ab30-f3fb8d2e7dce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:44:58.499473 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c33861-06fe-434f-ab30-f3fb8d2e7dce admin admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:44:58.499755 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c33861-06fe-434f-ab30-f3fb8d2e7dce admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:44:58.500030 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c33861-06fe-434f-ab30-f3fb8d2e7dce admin admin] PATH_INFO: `/v3/domains` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:44:58.500755 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07c33861-06fe-434f-ab30-f3fb8d2e7dce admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:44:58.501093 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07c33861-06fe-434f-ab30-f3fb8d2e7dce admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:44:58.502679 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07c33861-06fe-434f-ab30-f3fb8d2e7dce admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:44:58.514643 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1347/2685] 10.222.0.33 () {58 vars in 1152 bytes} [Mon Jul 25 05:44:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 05:44:58.518944 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe25c785-60cf-480f-a2f9-80dac486de1a None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:44:58.529034 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe25c785-60cf-480f-a2f9-80dac486de1a None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:44:58.533670 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f093ef5-5d11-45c5-a982-3f55d952e876 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:44:58.534318 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f093ef5-5d11-45c5-a982-3f55d952e876 admin admin] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:44:58.534611 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f093ef5-5d11-45c5-a982-3f55d952e876 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:44:58.534892 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f093ef5-5d11-45c5-a982-3f55d952e876 admin admin] PATH_INFO: `/v3/projects` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:44:58.535600 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f093ef5-5d11-45c5-a982-3f55d952e876 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:44:58.537534 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f093ef5-5d11-45c5-a982-3f55d952e876 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:44:58.563131 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1339/2686] 10.222.0.33 () {60 vars in 1152 bytes} [Mon Jul 25 05:44:58 2022] POST /identity/v3/projects => generated 367 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 05:44:58.567216 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c33861-06fe-434f-ab30-f3fb8d2e7dce admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:44:58.578116 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c33861-06fe-434f-ab30-f3fb8d2e7dce admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:44:58.636137 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ebf04f8-e365-47b6-868d-d354bdb644c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:44:58.636848 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ebf04f8-e365-47b6-868d-d354bdb644c4 admin admin] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:44:58.637236 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ebf04f8-e365-47b6-868d-d354bdb644c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:44:58.637575 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ebf04f8-e365-47b6-868d-d354bdb644c4 admin admin] PATH_INFO: `/v3/users` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:44:58.638338 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ebf04f8-e365-47b6-868d-d354bdb644c4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:44:58.640577 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ebf04f8-e365-47b6-868d-d354bdb644c4 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:44:58.661090 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1348/2687] 10.222.0.33 () {60 vars in 1143 bytes} [Mon Jul 25 05:44:58 2022] POST /identity/v3/users => generated 284 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 25 05:44:58.664866 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f093ef5-5d11-45c5-a982-3f55d952e876 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:44:58.675306 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f093ef5-5d11-45c5-a982-3f55d952e876 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:44:58.679908 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e46c8b53-74a1-49d9-9773-75b8bc2ad08a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:44:58.680563 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46c8b53-74a1-49d9-9773-75b8bc2ad08a admin admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:44:58.680880 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46c8b53-74a1-49d9-9773-75b8bc2ad08a admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:44:58.681159 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46c8b53-74a1-49d9-9773-75b8bc2ad08a admin admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:44:58.681886 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e46c8b53-74a1-49d9-9773-75b8bc2ad08a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:44:58.683478 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e46c8b53-74a1-49d9-9773-75b8bc2ad08a admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:44:58.694862 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1340/2688] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jul 25 05:44:58 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 05:44:58.699459 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ebf04f8-e365-47b6-868d-d354bdb644c4 admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:44:58.709446 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ebf04f8-e365-47b6-868d-d354bdb644c4 admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:44:58.715097 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab0c050e-1e59-40b9-a1bf-35615eb5cb4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:44:58.715755 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0c050e-1e59-40b9-a1bf-35615eb5cb4d admin admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:44:58.716046 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0c050e-1e59-40b9-a1bf-35615eb5cb4d admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:44:58.716331 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0c050e-1e59-40b9-a1bf-35615eb5cb4d admin admin] PATH_INFO: `/v3/roles` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:44:58.717106 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab0c050e-1e59-40b9-a1bf-35615eb5cb4d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:44:58.718716 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab0c050e-1e59-40b9-a1bf-35615eb5cb4d admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:44:58.729197 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1349/2689] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jul 25 05:44:58 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 05:44:58.733234 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e46c8b53-74a1-49d9-9773-75b8bc2ad08a admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:44:58.739394 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:44:58.743957 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e46c8b53-74a1-49d9-9773-75b8bc2ad08a admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:44:58.750157 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b7b4915-b96a-4577-9fad-42d3aab35346 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:44:58.751312 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7b4915-b96a-4577-9fad-42d3aab35346 admin admin] REQUEST_METHOD: `PUT` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:44:58.751767 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7b4915-b96a-4577-9fad-42d3aab35346 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:44:58.752127 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7b4915-b96a-4577-9fad-42d3aab35346 admin admin] PATH_INFO: `/v3/projects/5b5b5ea5ac42431485b5540bf9e7c45f/users/860339d8c0f64a639da4feb7cf3a26cf/roles/5074e74e35f54d4b9bb415c34999a82a` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:44:58.769052 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b7b4915-b96a-4577-9fad-42d3aab35346 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5b5b5ea5ac42431485b5540bf9e7c45f, user_id=860339d8c0f64a639da4feb7cf3a26cf, role_id=5074e74e35f54d4b9bb415c34999a82a)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:44:58.771683 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b7b4915-b96a-4577-9fad-42d3aab35346 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:44:58.785639 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1341/2690] 10.222.0.33 () {60 vars in 1482 bytes} [Mon Jul 25 05:44:58 2022] PUT /identity/v3/projects/5b5b5ea5ac42431485b5540bf9e7c45f/users/860339d8c0f64a639da4feb7cf3a26cf/roles/5074e74e35f54d4b9bb415c34999a82a => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 05:44:58.797095 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1dd0a0-900b-4799-a106-bf215698e750 None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:44:58.797483 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1dd0a0-900b-4799-a106-bf215698e750 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:44:58.797774 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1dd0a0-900b-4799-a106-bf215698e750 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:44:58.825522 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-4c1dd0a0-900b-4799-a106-bf215698e750 None None] MFA Rules not processed for user `860339d8c0f64a639da4feb7cf3a26cf`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:44:58.827041 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-4c1dd0a0-900b-4799-a106-bf215698e750 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:44:58.860697 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:44:58.898967 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1350/2691] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 25 05:44:58 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:44:58.904381 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b7b4915-b96a-4577-9fad-42d3aab35346 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:44:58.996183 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b7b4915-b96a-4577-9fad-42d3aab35346 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:44:59.001121 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216f32de-95c3-42b6-9f1b-0b871182b041 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:44:59.001799 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216f32de-95c3-42b6-9f1b-0b871182b041 admin admin] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:44:59.002083 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216f32de-95c3-42b6-9f1b-0b871182b041 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:44:59.002361 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216f32de-95c3-42b6-9f1b-0b871182b041 admin admin] PATH_INFO: `/v3/projects` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:44:59.003117 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-216f32de-95c3-42b6-9f1b-0b871182b041 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:44:59.005403 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-216f32de-95c3-42b6-9f1b-0b871182b041 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:44:59.036880 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1342/2692] 10.222.0.33 () {60 vars in 1152 bytes} [Mon Jul 25 05:44:58 2022] POST /identity/v3/projects => generated 367 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 05:44:59.041773 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1dd0a0-900b-4799-a106-bf215698e750 None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:44:59.053488 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1dd0a0-900b-4799-a106-bf215698e750 None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:44:59.127749 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ada0e4-674a-4ce1-9e2a-0122a75dd122 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:44:59.127749 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ada0e4-674a-4ce1-9e2a-0122a75dd122 admin admin] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:44:59.127749 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ada0e4-674a-4ce1-9e2a-0122a75dd122 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:44:59.127749 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ada0e4-674a-4ce1-9e2a-0122a75dd122 admin admin] PATH_INFO: `/v3/users` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:44:59.128280 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00ada0e4-674a-4ce1-9e2a-0122a75dd122 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:44:59.128280 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00ada0e4-674a-4ce1-9e2a-0122a75dd122 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:44:59.147613 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1351/2693] 10.222.0.33 () {60 vars in 1143 bytes} [Mon Jul 25 05:44:59 2022] POST /identity/v3/users => generated 290 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 25 05:44:59.152525 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216f32de-95c3-42b6-9f1b-0b871182b041 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:44:59.163268 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216f32de-95c3-42b6-9f1b-0b871182b041 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:44:59.168873 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce951b30-9020-4f2f-9222-e54cbf264b81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:44:59.169687 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce951b30-9020-4f2f-9222-e54cbf264b81 admin admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:44:59.170000 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce951b30-9020-4f2f-9222-e54cbf264b81 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:44:59.170281 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce951b30-9020-4f2f-9222-e54cbf264b81 admin admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:44:59.171021 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce951b30-9020-4f2f-9222-e54cbf264b81 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:44:59.172628 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce951b30-9020-4f2f-9222-e54cbf264b81 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:44:59.183627 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1343/2694] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jul 25 05:44:59 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 05:44:59.187974 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ada0e4-674a-4ce1-9e2a-0122a75dd122 admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:44:59.198115 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ada0e4-674a-4ce1-9e2a-0122a75dd122 admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:44:59.203262 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e581862-8d36-4600-b8df-24cf6aacfaab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:44:59.204221 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e581862-8d36-4600-b8df-24cf6aacfaab admin admin] REQUEST_METHOD: `PUT` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:44:59.204602 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e581862-8d36-4600-b8df-24cf6aacfaab admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:44:59.204982 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e581862-8d36-4600-b8df-24cf6aacfaab admin admin] PATH_INFO: `/v3/projects/c920754764474c08aca138f6adc74e4f/users/020f2b67ec8d41d7b0aafcf766b7e019/roles/39e5c666285a42fe8fc2c3afca6260d1` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:44:59.225510 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e581862-8d36-4600-b8df-24cf6aacfaab admin admin] RBAC: Authorizing `identity:create_grant(project_id=c920754764474c08aca138f6adc74e4f, user_id=020f2b67ec8d41d7b0aafcf766b7e019, role_id=39e5c666285a42fe8fc2c3afca6260d1)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:44:59.228330 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e581862-8d36-4600-b8df-24cf6aacfaab admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:44:59.242938 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1352/2695] 10.222.0.33 () {60 vars in 1482 bytes} [Mon Jul 25 05:44:59 2022] PUT /identity/v3/projects/c920754764474c08aca138f6adc74e4f/users/020f2b67ec8d41d7b0aafcf766b7e019/roles/39e5c666285a42fe8fc2c3afca6260d1 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 05:44:59.247366 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce951b30-9020-4f2f-9222-e54cbf264b81 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:44:59.257677 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce951b30-9020-4f2f-9222-e54cbf264b81 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:44:59.323235 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b0232c6-d535-4567-9a5e-86b24efb46b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:44:59.324053 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0232c6-d535-4567-9a5e-86b24efb46b0 admin admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:44:59.324405 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0232c6-d535-4567-9a5e-86b24efb46b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:44:59.324763 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0232c6-d535-4567-9a5e-86b24efb46b0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:44:59.325763 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b0232c6-d535-4567-9a5e-86b24efb46b0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:44:59.327612 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b0232c6-d535-4567-9a5e-86b24efb46b0 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:44:59.346756 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1344/2696] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jul 25 05:44:59 2022] GET /identity/v3/roles => generated 1420 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 05:44:59.351431 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e581862-8d36-4600-b8df-24cf6aacfaab admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:44:59.362689 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e581862-8d36-4600-b8df-24cf6aacfaab admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:44:59.367244 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d97175-177a-4ac8-8a80-ebbab9784352 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:44:59.368123 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d97175-177a-4ac8-8a80-ebbab9784352 admin admin] REQUEST_METHOD: `PUT` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:44:59.368679 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d97175-177a-4ac8-8a80-ebbab9784352 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:44:59.369028 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d97175-177a-4ac8-8a80-ebbab9784352 admin admin] PATH_INFO: `/v3/projects/c920754764474c08aca138f6adc74e4f/users/020f2b67ec8d41d7b0aafcf766b7e019/roles/39e5c666285a42fe8fc2c3afca6260d1` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:44:59.371040 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07d97175-177a-4ac8-8a80-ebbab9784352 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c920754764474c08aca138f6adc74e4f, user_id=020f2b67ec8d41d7b0aafcf766b7e019, role_id=39e5c666285a42fe8fc2c3afca6260d1)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:44:59.373675 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07d97175-177a-4ac8-8a80-ebbab9784352 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:44:59.385070 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1353/2697] 10.222.0.33 () {60 vars in 1482 bytes} [Mon Jul 25 05:44:59 2022] PUT /identity/v3/projects/c920754764474c08aca138f6adc74e4f/users/020f2b67ec8d41d7b0aafcf766b7e019/roles/39e5c666285a42fe8fc2c3afca6260d1 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 05:44:59.394971 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb40ad8-21bf-4a8a-9811-98afcbc997bf None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:44:59.395258 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb40ad8-21bf-4a8a-9811-98afcbc997bf None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:44:59.395535 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb40ad8-21bf-4a8a-9811-98afcbc997bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:44:59.421828 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-4bb40ad8-21bf-4a8a-9811-98afcbc997bf None None] MFA Rules not processed for user `020f2b67ec8d41d7b0aafcf766b7e019`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:44:59.423152 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-4bb40ad8-21bf-4a8a-9811-98afcbc997bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:44:59.465479 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:44:59.503818 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1345/2698] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 25 05:44:59 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:44:59.514902 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d97175-177a-4ac8-8a80-ebbab9784352 admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:44:59.516668 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-07d97175-177a-4ac8-8a80-ebbab9784352 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:44:59.578764 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d97175-177a-4ac8-8a80-ebbab9784352 admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:44:59.590179 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-688512bd-f799-403e-9c71-c34cca3be8d3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db57b478c25643dbaf936f90f6e17d0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92afba266cce4d1bad4134c351e74be3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:44:59.590919 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688512bd-f799-403e-9c71-c34cca3be8d3 service cinder] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:44:59.591359 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688512bd-f799-403e-9c71-c34cca3be8d3 service cinder] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:44:59.591782 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688512bd-f799-403e-9c71-c34cca3be8d3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:44:59.604378 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-688512bd-f799-403e-9c71-c34cca3be8d3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:44:59.605945 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-688512bd-f799-403e-9c71-c34cca3be8d3 service cinder] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:44:59.616441 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1354/2699] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jul 25 05:44:59 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 05:44:59.718448 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb40ad8-21bf-4a8a-9811-98afcbc997bf None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:44:59.720438 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-4bb40ad8-21bf-4a8a-9811-98afcbc997bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:44:59.737052 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb40ad8-21bf-4a8a-9811-98afcbc997bf None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:44:59.740862 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff1fcf8-57aa-4ff1-a89c-a1475d39228d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db57b478c25643dbaf936f90f6e17d0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92afba266cce4d1bad4134c351e74be3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:44:59.741648 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff1fcf8-57aa-4ff1-a89c-a1475d39228d service cinder] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:44:59.742000 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff1fcf8-57aa-4ff1-a89c-a1475d39228d service cinder] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:44:59.742345 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff1fcf8-57aa-4ff1-a89c-a1475d39228d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:44:59.802340 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eff1fcf8-57aa-4ff1-a89c-a1475d39228d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:44:59.803798 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eff1fcf8-57aa-4ff1-a89c-a1475d39228d service cinder] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:44:59.813981 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1346/2700] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jul 25 05:44:59 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 05:45:01.618473 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-688512bd-f799-403e-9c71-c34cca3be8d3 service cinder] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:45:01.627256 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-688512bd-f799-403e-9c71-c34cca3be8d3 service cinder] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:45:01.630946 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de39455-f8d5-4f8b-b764-358794cbec3f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db57b478c25643dbaf936f90f6e17d0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92afba266cce4d1bad4134c351e74be3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:45:01.631582 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de39455-f8d5-4f8b-b764-358794cbec3f service cinder] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:45:01.631870 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de39455-f8d5-4f8b-b764-358794cbec3f service cinder] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:45:01.632152 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de39455-f8d5-4f8b-b764-358794cbec3f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:45:01.634790 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-5de39455-f8d5-4f8b-b764-358794cbec3f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:45:01.660737 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:45:01.695736 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5de39455-f8d5-4f8b-b764-358794cbec3f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:45:01.697963 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5de39455-f8d5-4f8b-b764-358794cbec3f service cinder] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:45:01.706877 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1355/2701] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jul 25 05:45:01 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 05:45:05.588703 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff1fcf8-57aa-4ff1-a89c-a1475d39228d service cinder] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:45:05.708836 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff1fcf8-57aa-4ff1-a89c-a1475d39228d service cinder] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:45:05.714198 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b910aa-80b0-4b0c-aa34-cf3a06e64417 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:45:05.715074 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b910aa-80b0-4b0c-aa34-cf3a06e64417 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:45:05.715362 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b910aa-80b0-4b0c-aa34-cf3a06e64417 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:45:05.715649 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b910aa-80b0-4b0c-aa34-cf3a06e64417 admin admin] PATH_INFO: `/v3/users/860339d8c0f64a639da4feb7cf3a26cf` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:45:05.717005 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4b910aa-80b0-4b0c-aa34-cf3a06e64417 admin admin] RBAC: Authorizing `identity:delete_user(user_id=860339d8c0f64a639da4feb7cf3a26cf)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:45:05.718933 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4b910aa-80b0-4b0c-aa34-cf3a06e64417 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:45:05.759689 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:45:05.772988 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-a4b910aa-80b0-4b0c-aa34-cf3a06e64417 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '860339d8c0f64a639da4feb7cf3a26cf'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:45:05.776925 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-a4b910aa-80b0-4b0c-aa34-cf3a06e64417 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '860339d8c0f64a639da4feb7cf3a26cf'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:45:05.785041 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-a4b910aa-80b0-4b0c-aa34-cf3a06e64417 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '860339d8c0f64a639da4feb7cf3a26cf'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:45:05.811533 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-a4b910aa-80b0-4b0c-aa34-cf3a06e64417 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '860339d8c0f64a639da4feb7cf3a26cf'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:45:05.812302 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-a4b910aa-80b0-4b0c-aa34-cf3a06e64417 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '860339d8c0f64a639da4feb7cf3a26cf'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:45:05.826168 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1347/2702] 10.222.0.33 () {58 vars in 1223 bytes} [Mon Jul 25 05:45:05 2022] DELETE /identity/v3/users/860339d8c0f64a639da4feb7cf3a26cf => generated 0 bytes in 239 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 05:45:05.830831 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75c7221-c52a-4996-9500-4f70da661e59 None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:45:05.831317 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75c7221-c52a-4996-9500-4f70da661e59 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:45:05.831466 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75c7221-c52a-4996-9500-4f70da661e59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:45:05.884188 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-f75c7221-c52a-4996-9500-4f70da661e59 None None] MFA Rules not processed for user `1eae540242d94c10b0ad33960a855f08`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:45:05.885497 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-f75c7221-c52a-4996-9500-4f70da661e59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:45:05.945513 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1356/2703] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 25 05:45:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:45:05.956223 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b910aa-80b0-4b0c-aa34-cf3a06e64417 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:45:05.957716 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-a4b910aa-80b0-4b0c-aa34-cf3a06e64417 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:45:06.051173 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b910aa-80b0-4b0c-aa34-cf3a06e64417 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:45:06.055308 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1011023c-b57d-4dde-818a-1cc82ecacafb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b2931200f99e43d79a50a9f6e02738e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92afba266cce4d1bad4134c351e74be3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:45:06.057002 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1011023c-b57d-4dde-818a-1cc82ecacafb service neutron] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:45:06.057424 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1011023c-b57d-4dde-818a-1cc82ecacafb service neutron] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:45:06.057830 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1011023c-b57d-4dde-818a-1cc82ecacafb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:45:06.069314 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1011023c-b57d-4dde-818a-1cc82ecacafb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:45:06.071536 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1011023c-b57d-4dde-818a-1cc82ecacafb service neutron] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:45:06.097662 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1348/2704] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jul 25 05:45:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 05:45:06.546143 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f75c7221-c52a-4996-9500-4f70da661e59 None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:45:06.547690 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-f75c7221-c52a-4996-9500-4f70da661e59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:45:06.566848 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f75c7221-c52a-4996-9500-4f70da661e59 None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:45:06.575398 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89efeb5b-dbe9-4f6a-a056-80843a40c561 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:45:06.592802 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89efeb5b-dbe9-4f6a-a056-80843a40c561 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:45:06.594914 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89efeb5b-dbe9-4f6a-a056-80843a40c561 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:45:06.596089 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89efeb5b-dbe9-4f6a-a056-80843a40c561 admin admin] PATH_INFO: `/v3/projects/5b5b5ea5ac42431485b5540bf9e7c45f` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:45:06.598761 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89efeb5b-dbe9-4f6a-a056-80843a40c561 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5b5b5ea5ac42431485b5540bf9e7c45f)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:45:06.601637 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89efeb5b-dbe9-4f6a-a056-80843a40c561 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:45:06.706386 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-89efeb5b-dbe9-4f6a-a056-80843a40c561 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5b5b5ea5ac42431485b5540bf9e7c45f', 'request_id': 'req-89efeb5b-dbe9-4f6a-a056-80843a40c561'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:45:06.736703 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-89efeb5b-dbe9-4f6a-a056-80843a40c561 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5b5b5ea5ac42431485b5540bf9e7c45f', 'request_id': 'req-89efeb5b-dbe9-4f6a-a056-80843a40c561'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:45:06.756126 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-89efeb5b-dbe9-4f6a-a056-80843a40c561 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5b5b5ea5ac42431485b5540bf9e7c45f', 'request_id': 'req-89efeb5b-dbe9-4f6a-a056-80843a40c561'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:45:06.782755 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-89efeb5b-dbe9-4f6a-a056-80843a40c561 admin admin] The token cache is being invalidate because project 5b5b5ea5ac42431485b5540bf9e7c45f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80248) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 05:45:06.783231 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-89efeb5b-dbe9-4f6a-a056-80843a40c561 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:45:06.783911 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-89efeb5b-dbe9-4f6a-a056-80843a40c561 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:45:06.785701 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1357/2705] 10.222.0.33 () {58 vars in 1232 bytes} [Mon Jul 25 05:45:06 2022] DELETE /identity/v3/projects/5b5b5ea5ac42431485b5540bf9e7c45f => generated 0 bytes in 240 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 05:45:06.789567 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1011023c-b57d-4dde-818a-1cc82ecacafb service neutron] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:45:06.802865 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1011023c-b57d-4dde-818a-1cc82ecacafb service neutron] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:45:06.901850 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e07c9af7-5244-41bc-84d2-97a7ec00b76a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:45:06.902742 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07c9af7-5244-41bc-84d2-97a7ec00b76a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:45:06.903031 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07c9af7-5244-41bc-84d2-97a7ec00b76a admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:45:06.903313 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07c9af7-5244-41bc-84d2-97a7ec00b76a admin admin] PATH_INFO: `/v3/users/020f2b67ec8d41d7b0aafcf766b7e019` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:45:06.904542 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e07c9af7-5244-41bc-84d2-97a7ec00b76a admin admin] RBAC: Authorizing `identity:delete_user(user_id=020f2b67ec8d41d7b0aafcf766b7e019)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:45:06.906507 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e07c9af7-5244-41bc-84d2-97a7ec00b76a admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:45:06.951782 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:45:06.957956 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-e07c9af7-5244-41bc-84d2-97a7ec00b76a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '020f2b67ec8d41d7b0aafcf766b7e019'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:45:06.959006 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-e07c9af7-5244-41bc-84d2-97a7ec00b76a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '020f2b67ec8d41d7b0aafcf766b7e019'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:45:06.967600 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-e07c9af7-5244-41bc-84d2-97a7ec00b76a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '020f2b67ec8d41d7b0aafcf766b7e019'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:45:06.980154 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-e07c9af7-5244-41bc-84d2-97a7ec00b76a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '020f2b67ec8d41d7b0aafcf766b7e019'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:45:06.980865 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-e07c9af7-5244-41bc-84d2-97a7ec00b76a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '020f2b67ec8d41d7b0aafcf766b7e019'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:45:06.997713 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1349/2706] 10.222.0.33 () {58 vars in 1223 bytes} [Mon Jul 25 05:45:06 2022] DELETE /identity/v3/users/020f2b67ec8d41d7b0aafcf766b7e019 => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 25 05:45:07.280008 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89efeb5b-dbe9-4f6a-a056-80843a40c561 admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:45:07.281576 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-89efeb5b-dbe9-4f6a-a056-80843a40c561 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:45:07.298648 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89efeb5b-dbe9-4f6a-a056-80843a40c561 admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:45:07.325381 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:45:07.385653 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5067d32e-5302-47ca-8b27-6c21cf31948d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:45:07.386544 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5067d32e-5302-47ca-8b27-6c21cf31948d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:45:07.386832 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5067d32e-5302-47ca-8b27-6c21cf31948d admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:45:07.387118 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5067d32e-5302-47ca-8b27-6c21cf31948d admin admin] PATH_INFO: `/v3/projects/c920754764474c08aca138f6adc74e4f` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:45:07.388326 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5067d32e-5302-47ca-8b27-6c21cf31948d admin admin] RBAC: Authorizing `identity:delete_project(project_id=c920754764474c08aca138f6adc74e4f)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:45:07.390276 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5067d32e-5302-47ca-8b27-6c21cf31948d admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:45:07.445007 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-5067d32e-5302-47ca-8b27-6c21cf31948d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c920754764474c08aca138f6adc74e4f', 'request_id': 'req-5067d32e-5302-47ca-8b27-6c21cf31948d'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:45:07.451127 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-5067d32e-5302-47ca-8b27-6c21cf31948d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c920754764474c08aca138f6adc74e4f', 'request_id': 'req-5067d32e-5302-47ca-8b27-6c21cf31948d'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:45:07.457197 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-5067d32e-5302-47ca-8b27-6c21cf31948d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c920754764474c08aca138f6adc74e4f', 'request_id': 'req-5067d32e-5302-47ca-8b27-6c21cf31948d'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:45:07.470658 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-5067d32e-5302-47ca-8b27-6c21cf31948d admin admin] The token cache is being invalidate because project c920754764474c08aca138f6adc74e4f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80248) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 05:45:07.470948 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-5067d32e-5302-47ca-8b27-6c21cf31948d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:45:07.471582 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-5067d32e-5302-47ca-8b27-6c21cf31948d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:45:07.473192 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1358/2707] 10.222.0.33 () {58 vars in 1232 bytes} [Mon Jul 25 05:45:07 2022] DELETE /identity/v3/projects/c920754764474c08aca138f6adc74e4f => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 05:45:07.479232 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5bf13d-84a1-479d-8f63-8b441b9e48bb None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:45:07.479533 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5bf13d-84a1-479d-8f63-8b441b9e48bb None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:45:07.479816 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5bf13d-84a1-479d-8f63-8b441b9e48bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:45:07.527449 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-2e5bf13d-84a1-479d-8f63-8b441b9e48bb None None] MFA Rules not processed for user `1eae540242d94c10b0ad33960a855f08`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:45:07.528601 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-2e5bf13d-84a1-479d-8f63-8b441b9e48bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:45:07.587419 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1350/2708] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 25 05:45:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:45:07.596029 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1f01ab-ddc1-4191-a18e-745c4cbc0bbf None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:45:07.596314 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1f01ab-ddc1-4191-a18e-745c4cbc0bbf None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:45:07.596586 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1f01ab-ddc1-4191-a18e-745c4cbc0bbf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:45:07.615147 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-7a1f01ab-ddc1-4191-a18e-745c4cbc0bbf None None] MFA Rules not processed for user `1eae540242d94c10b0ad33960a855f08`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:45:07.616339 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-7a1f01ab-ddc1-4191-a18e-745c4cbc0bbf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:45:07.625626 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1359/2709] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 25 05:45:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:45:07.629682 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e5bf13d-84a1-479d-8f63-8b441b9e48bb None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:45:07.639273 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e5bf13d-84a1-479d-8f63-8b441b9e48bb None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:45:07.643895 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a44e03c-f187-46af-a188-a90e784f813f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:45:07.644563 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a44e03c-f187-46af-a188-a90e784f813f admin admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:45:07.644875 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a44e03c-f187-46af-a188-a90e784f813f admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:45:07.645155 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a44e03c-f187-46af-a188-a90e784f813f admin admin] PATH_INFO: `/v3/domains` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:45:07.645893 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a44e03c-f187-46af-a188-a90e784f813f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:45:07.646201 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a44e03c-f187-46af-a188-a90e784f813f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:45:07.647793 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a44e03c-f187-46af-a188-a90e784f813f admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:45:07.659230 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1351/2710] 10.222.0.33 () {58 vars in 1152 bytes} [Mon Jul 25 05:45:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:45:07.663930 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a1f01ab-ddc1-4191-a18e-745c4cbc0bbf None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:45:07.673798 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a1f01ab-ddc1-4191-a18e-745c4cbc0bbf None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:45:07.678614 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f174cbe0-c592-43da-adb3-a8a34bfebad4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:45:07.679255 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f174cbe0-c592-43da-adb3-a8a34bfebad4 admin admin] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:45:07.679540 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f174cbe0-c592-43da-adb3-a8a34bfebad4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:45:07.679816 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f174cbe0-c592-43da-adb3-a8a34bfebad4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:45:07.680513 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f174cbe0-c592-43da-adb3-a8a34bfebad4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:45:07.683199 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f174cbe0-c592-43da-adb3-a8a34bfebad4 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:45:07.709862 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1360/2711] 10.222.0.33 () {60 vars in 1152 bytes} [Mon Jul 25 05:45:07 2022] POST /identity/v3/projects => generated 383 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 05:45:07.714326 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a44e03c-f187-46af-a188-a90e784f813f admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:45:07.724178 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a44e03c-f187-46af-a188-a90e784f813f admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:45:07.781608 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb6f719-19e5-4fb7-bb1f-62343d229f98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:45:07.782194 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb6f719-19e5-4fb7-bb1f-62343d229f98 admin admin] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:45:07.782484 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb6f719-19e5-4fb7-bb1f-62343d229f98 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:45:07.782758 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb6f719-19e5-4fb7-bb1f-62343d229f98 admin admin] PATH_INFO: `/v3/users` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:45:07.783458 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bb6f719-19e5-4fb7-bb1f-62343d229f98 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:45:07.785390 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bb6f719-19e5-4fb7-bb1f-62343d229f98 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:45:07.805351 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1352/2712] 10.222.0.33 () {60 vars in 1143 bytes} [Mon Jul 25 05:45:07 2022] POST /identity/v3/users => generated 292 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 05:45:07.810473 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f174cbe0-c592-43da-adb3-a8a34bfebad4 admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:45:07.822839 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f174cbe0-c592-43da-adb3-a8a34bfebad4 admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:45:07.827889 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a64a74-91eb-4eb8-b407-8e15acf72048 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:45:07.828458 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a64a74-91eb-4eb8-b407-8e15acf72048 admin admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:45:07.828743 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a64a74-91eb-4eb8-b407-8e15acf72048 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:45:07.829056 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a64a74-91eb-4eb8-b407-8e15acf72048 admin admin] PATH_INFO: `/v3/roles` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:45:07.829902 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5a64a74-91eb-4eb8-b407-8e15acf72048 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:45:07.831354 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5a64a74-91eb-4eb8-b407-8e15acf72048 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:45:07.842226 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1361/2713] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jul 25 05:45:07 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 05:45:07.851812 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb6f719-19e5-4fb7-bb1f-62343d229f98 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:45:07.857649 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb6f719-19e5-4fb7-bb1f-62343d229f98 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:45:07.862217 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2380c8b-50cc-4670-b81d-669a965bbcdf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:45:07.862870 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2380c8b-50cc-4670-b81d-669a965bbcdf admin admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:45:07.863157 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2380c8b-50cc-4670-b81d-669a965bbcdf admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:45:07.863433 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2380c8b-50cc-4670-b81d-669a965bbcdf admin admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:45:07.864160 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2380c8b-50cc-4670-b81d-669a965bbcdf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:45:07.865768 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2380c8b-50cc-4670-b81d-669a965bbcdf admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:45:07.871648 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=92,keystone:DELETE=4,keystone:INSERT=4 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:45:07.877790 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1353/2714] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jul 25 05:45:07 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 05:45:07.881943 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a64a74-91eb-4eb8-b407-8e15acf72048 admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:45:07.891875 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a64a74-91eb-4eb8-b407-8e15acf72048 admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:45:07.896507 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b0ca381-3fb8-4a96-bb38-ffd589df75cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:45:07.897548 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0ca381-3fb8-4a96-bb38-ffd589df75cd admin admin] REQUEST_METHOD: `PUT` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:45:07.897759 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0ca381-3fb8-4a96-bb38-ffd589df75cd admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:45:07.897946 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0ca381-3fb8-4a96-bb38-ffd589df75cd admin admin] PATH_INFO: `/v3/projects/9e542cc2bf96413a85d2eeb1776145ab/users/5b0f119c38564a499386edbd3673c5b6/roles/5074e74e35f54d4b9bb415c34999a82a` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:45:07.914046 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b0ca381-3fb8-4a96-bb38-ffd589df75cd admin admin] RBAC: Authorizing `identity:create_grant(project_id=9e542cc2bf96413a85d2eeb1776145ab, user_id=5b0f119c38564a499386edbd3673c5b6, role_id=5074e74e35f54d4b9bb415c34999a82a)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:45:07.917646 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b0ca381-3fb8-4a96-bb38-ffd589df75cd admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:45:07.931761 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1362/2715] 10.222.0.33 () {60 vars in 1482 bytes} [Mon Jul 25 05:45:07 2022] PUT /identity/v3/projects/9e542cc2bf96413a85d2eeb1776145ab/users/5b0f119c38564a499386edbd3673c5b6/roles/5074e74e35f54d4b9bb415c34999a82a => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 05:45:07.943573 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db68b6bd-08e3-4141-b798-602248396256 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:45:07.943573 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db68b6bd-08e3-4141-b798-602248396256 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:45:07.943573 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db68b6bd-08e3-4141-b798-602248396256 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:45:07.971503 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-db68b6bd-08e3-4141-b798-602248396256 None None] MFA Rules not processed for user `5b0f119c38564a499386edbd3673c5b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:45:07.972734 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-db68b6bd-08e3-4141-b798-602248396256 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:45:08.005500 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad91b4c-9280-44b2-b59e-040727848796 None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:45:08.005798 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad91b4c-9280-44b2-b59e-040727848796 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:45:08.006084 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad91b4c-9280-44b2-b59e-040727848796 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:45:08.034446 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-cad91b4c-9280-44b2-b59e-040727848796 None None] MFA Rules not processed for user `3352531be1b54f76b864383ccc2e5ec2`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:45:08.035566 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-cad91b4c-9280-44b2-b59e-040727848796 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:45:08.039918 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1354/2716] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 25 05:45:07 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:45:08.044540 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db68b6bd-08e3-4141-b798-602248396256 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:45:08.069734 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:45:08.101633 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1363/2717] 10.222.0.33 () {58 vars in 967 bytes} [Mon Jul 25 05:45:08 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 05:45:08.108266 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db68b6bd-08e3-4141-b798-602248396256 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:45:08.113568 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e90953a-6daf-42ec-9c31-5233f39fb16b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:45:08.113812 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad91b4c-9280-44b2-b59e-040727848796 None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:45:08.114232 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e90953a-6daf-42ec-9c31-5233f39fb16b admin admin] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:45:08.114518 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e90953a-6daf-42ec-9c31-5233f39fb16b admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:45:08.114797 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e90953a-6daf-42ec-9c31-5233f39fb16b admin admin] PATH_INFO: `/v3/projects` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:45:08.115256 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-cad91b4c-9280-44b2-b59e-040727848796 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:45:08.115485 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e90953a-6daf-42ec-9c31-5233f39fb16b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:45:08.117450 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e90953a-6daf-42ec-9c31-5233f39fb16b admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:45:08.147567 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1355/2718] 10.222.0.33 () {60 vars in 1152 bytes} [Mon Jul 25 05:45:08 2022] POST /identity/v3/projects => generated 385 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 25 05:45:08.148553 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e90953a-6daf-42ec-9c31-5233f39fb16b admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:45:08.196932 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad91b4c-9280-44b2-b59e-040727848796 None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:45:08.214253 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e90953a-6daf-42ec-9c31-5233f39fb16b admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:45:08.218996 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d0ac944-5e56-4b56-93ab-1dbcb8020261 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:45:08.219670 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0ac944-5e56-4b56-93ab-1dbcb8020261 admin admin] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:45:08.219960 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0ac944-5e56-4b56-93ab-1dbcb8020261 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:45:08.220243 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0ac944-5e56-4b56-93ab-1dbcb8020261 admin admin] PATH_INFO: `/v3/users` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:45:08.220988 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d0ac944-5e56-4b56-93ab-1dbcb8020261 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:45:08.222993 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d0ac944-5e56-4b56-93ab-1dbcb8020261 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:45:08.244991 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1356/2719] 10.222.0.33 () {60 vars in 1143 bytes} [Mon Jul 25 05:45:08 2022] POST /identity/v3/users => generated 299 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 25 05:45:08.248801 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d0ac944-5e56-4b56-93ab-1dbcb8020261 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:45:08.254890 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:45:08.258901 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d0ac944-5e56-4b56-93ab-1dbcb8020261 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:45:08.263469 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bcf0336-9b49-4bde-87a9-cef7558c6636 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb135a8d5d7444cfbfa336115ba2ba44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92afba266cce4d1bad4134c351e74be3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:45:08.264209 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bcf0336-9b49-4bde-87a9-cef7558c6636 service swift] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:45:08.264496 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bcf0336-9b49-4bde-87a9-cef7558c6636 service swift] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:45:08.264796 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bcf0336-9b49-4bde-87a9-cef7558c6636 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:45:08.265463 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4112274e-7165-4316-b199-1f3a007ff52c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:45:08.266243 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4112274e-7165-4316-b199-1f3a007ff52c admin admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:45:08.266564 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4112274e-7165-4316-b199-1f3a007ff52c admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:45:08.266878 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4112274e-7165-4316-b199-1f3a007ff52c admin admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:45:08.267638 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4112274e-7165-4316-b199-1f3a007ff52c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:45:08.269564 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4112274e-7165-4316-b199-1f3a007ff52c admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:45:08.283848 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1357/2720] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jul 25 05:45:08 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 25 05:45:08.284846 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4112274e-7165-4316-b199-1f3a007ff52c admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:45:08.294770 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4112274e-7165-4316-b199-1f3a007ff52c admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:45:08.299536 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf7e5f90-221b-4d46-a5af-f92ca854bf31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:45:08.300400 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7e5f90-221b-4d46-a5af-f92ca854bf31 admin admin] REQUEST_METHOD: `PUT` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:45:08.300689 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7e5f90-221b-4d46-a5af-f92ca854bf31 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:45:08.300999 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7e5f90-221b-4d46-a5af-f92ca854bf31 admin admin] PATH_INFO: `/v3/projects/bf1aa3e44dc54046853fe847d0b37325/users/6e7dd002675043a8a384e2478ae0ca2c/roles/39e5c666285a42fe8fc2c3afca6260d1` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:45:08.315748 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf7e5f90-221b-4d46-a5af-f92ca854bf31 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bf1aa3e44dc54046853fe847d0b37325, user_id=6e7dd002675043a8a384e2478ae0ca2c, role_id=39e5c666285a42fe8fc2c3afca6260d1)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:45:08.318540 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf7e5f90-221b-4d46-a5af-f92ca854bf31 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:45:08.320445 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bcf0336-9b49-4bde-87a9-cef7558c6636 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:45:08.322569 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bcf0336-9b49-4bde-87a9-cef7558c6636 service swift] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:45:08.331061 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1364/2721] 10.222.0.33 () {60 vars in 1384 bytes} [Mon Jul 25 05:45:08 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 218 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jul 25 05:45:08.333106 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1358/2722] 10.222.0.33 () {60 vars in 1482 bytes} [Mon Jul 25 05:45:08 2022] PUT /identity/v3/projects/bf1aa3e44dc54046853fe847d0b37325/users/6e7dd002675043a8a384e2478ae0ca2c/roles/39e5c666285a42fe8fc2c3afca6260d1 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 05:45:08.336920 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bcf0336-9b49-4bde-87a9-cef7558c6636 service swift] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:45:08.347598 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bcf0336-9b49-4bde-87a9-cef7558c6636 service swift] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:45:08.368173 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:45:08.411063 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf466dd-064c-4cb0-ac2d-cab943a6a0ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:45:08.411743 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf466dd-064c-4cb0-ac2d-cab943a6a0ce admin admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:45:08.412026 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf466dd-064c-4cb0-ac2d-cab943a6a0ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:45:08.412297 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf466dd-064c-4cb0-ac2d-cab943a6a0ce admin admin] PATH_INFO: `/v3/roles` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:45:08.413053 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bf466dd-064c-4cb0-ac2d-cab943a6a0ce admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:45:08.414666 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bf466dd-064c-4cb0-ac2d-cab943a6a0ce admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:45:08.426303 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1365/2723] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jul 25 05:45:08 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 05:45:08.430580 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf7e5f90-221b-4d46-a5af-f92ca854bf31 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:45:08.442792 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf7e5f90-221b-4d46-a5af-f92ca854bf31 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:45:08.447966 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5db8e05-0d3c-4ccc-a3dd-06875e3cc9aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:45:08.449000 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5db8e05-0d3c-4ccc-a3dd-06875e3cc9aa admin admin] REQUEST_METHOD: `PUT` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:45:08.449285 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5db8e05-0d3c-4ccc-a3dd-06875e3cc9aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:45:08.449562 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5db8e05-0d3c-4ccc-a3dd-06875e3cc9aa admin admin] PATH_INFO: `/v3/projects/bf1aa3e44dc54046853fe847d0b37325/users/6e7dd002675043a8a384e2478ae0ca2c/roles/39e5c666285a42fe8fc2c3afca6260d1` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:45:08.451555 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5db8e05-0d3c-4ccc-a3dd-06875e3cc9aa admin admin] RBAC: Authorizing `identity:create_grant(project_id=bf1aa3e44dc54046853fe847d0b37325, user_id=6e7dd002675043a8a384e2478ae0ca2c, role_id=39e5c666285a42fe8fc2c3afca6260d1)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:45:08.454322 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5db8e05-0d3c-4ccc-a3dd-06875e3cc9aa admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:45:08.465491 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1359/2724] 10.222.0.33 () {60 vars in 1482 bytes} [Mon Jul 25 05:45:08 2022] PUT /identity/v3/projects/bf1aa3e44dc54046853fe847d0b37325/users/6e7dd002675043a8a384e2478ae0ca2c/roles/39e5c666285a42fe8fc2c3afca6260d1 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 05:45:08.478615 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0285f93-fb5e-4223-bb06-e04cb26a7b93 None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:45:08.478932 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0285f93-fb5e-4223-bb06-e04cb26a7b93 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:45:08.479510 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0285f93-fb5e-4223-bb06-e04cb26a7b93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:45:08.512075 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-c0285f93-fb5e-4223-bb06-e04cb26a7b93 None None] MFA Rules not processed for user `6e7dd002675043a8a384e2478ae0ca2c`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:45:08.513261 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-c0285f93-fb5e-4223-bb06-e04cb26a7b93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:45:08.621234 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1366/2725] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 25 05:45:08 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:45:08.635699 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5db8e05-0d3c-4ccc-a3dd-06875e3cc9aa admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:45:08.637830 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-a5db8e05-0d3c-4ccc-a3dd-06875e3cc9aa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:45:08.706840 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5db8e05-0d3c-4ccc-a3dd-06875e3cc9aa admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:45:08.719023 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f168556-9e5f-414f-b531-436b9d498b22 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db57b478c25643dbaf936f90f6e17d0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92afba266cce4d1bad4134c351e74be3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:45:08.719775 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f168556-9e5f-414f-b531-436b9d498b22 service cinder] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:45:08.720826 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f168556-9e5f-414f-b531-436b9d498b22 service cinder] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:45:08.720826 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f168556-9e5f-414f-b531-436b9d498b22 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:45:08.780118 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f168556-9e5f-414f-b531-436b9d498b22 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:45:08.781642 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f168556-9e5f-414f-b531-436b9d498b22 service cinder] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:45:08.795125 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1360/2726] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jul 25 05:45:08 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 160 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 05:45:09.773209 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c33a720-a1ca-4ad7-8d9f-e6525da4869b None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:45:09.773665 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c33a720-a1ca-4ad7-8d9f-e6525da4869b None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:45:09.773840 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c33a720-a1ca-4ad7-8d9f-e6525da4869b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:45:09.801204 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-4c33a720-a1ca-4ad7-8d9f-e6525da4869b None None] MFA Rules not processed for user `1f815f111bf74739969736acea131927`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:45:09.804056 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-4c33a720-a1ca-4ad7-8d9f-e6525da4869b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:45:09.906136 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1367/2727] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 25 05:45:09 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:45:09.951558 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f168556-9e5f-414f-b531-436b9d498b22 service cinder] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:45:09.953367 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-2f168556-9e5f-414f-b531-436b9d498b22 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:45:10.033404 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f168556-9e5f-414f-b531-436b9d498b22 service cinder] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:45:10.041270 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7120f1df-8709-4ee9-a095-e471d3c395cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:45:10.042528 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7120f1df-8709-4ee9-a095-e471d3c395cd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:45:10.042973 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7120f1df-8709-4ee9-a095-e471d3c395cd admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:45:10.043430 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7120f1df-8709-4ee9-a095-e471d3c395cd admin admin] PATH_INFO: `/v3/users/284984d3c83f443eb3e17a9647c28f9a` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:45:10.045191 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7120f1df-8709-4ee9-a095-e471d3c395cd admin admin] RBAC: Authorizing `identity:delete_user(user_id=284984d3c83f443eb3e17a9647c28f9a)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:45:10.048481 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7120f1df-8709-4ee9-a095-e471d3c395cd admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:45:10.065388 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=97,keystone:INSERT=2,keystone:DELETE=1 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:45:10.109993 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-7120f1df-8709-4ee9-a095-e471d3c395cd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '284984d3c83f443eb3e17a9647c28f9a'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:45:10.110667 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-7120f1df-8709-4ee9-a095-e471d3c395cd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '284984d3c83f443eb3e17a9647c28f9a'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:45:10.117926 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-7120f1df-8709-4ee9-a095-e471d3c395cd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '284984d3c83f443eb3e17a9647c28f9a'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:45:10.131705 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-7120f1df-8709-4ee9-a095-e471d3c395cd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '284984d3c83f443eb3e17a9647c28f9a'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:45:10.132620 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-7120f1df-8709-4ee9-a095-e471d3c395cd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '284984d3c83f443eb3e17a9647c28f9a'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:45:10.148010 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1361/2728] 10.222.0.33 () {58 vars in 1223 bytes} [Mon Jul 25 05:45:09 2022] DELETE /identity/v3/users/284984d3c83f443eb3e17a9647c28f9a => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 05:45:10.155387 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90eb812f-0917-42f6-bdcf-79220899b587 None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:45:10.155387 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90eb812f-0917-42f6-bdcf-79220899b587 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:45:10.155387 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90eb812f-0917-42f6-bdcf-79220899b587 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:45:10.174580 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-90eb812f-0917-42f6-bdcf-79220899b587 None None] MFA Rules not processed for user `1eae540242d94c10b0ad33960a855f08`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:45:10.175997 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-90eb812f-0917-42f6-bdcf-79220899b587 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:45:10.228369 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:45:10.248905 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1368/2729] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 25 05:45:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 05:45:10.261795 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7120f1df-8709-4ee9-a095-e471d3c395cd admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:45:10.263556 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-7120f1df-8709-4ee9-a095-e471d3c395cd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:45:10.326967 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7120f1df-8709-4ee9-a095-e471d3c395cd admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:45:10.330831 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4575f564-5a29-4aa8-85cd-61623455dc72 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b2931200f99e43d79a50a9f6e02738e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92afba266cce4d1bad4134c351e74be3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:45:10.331604 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4575f564-5a29-4aa8-85cd-61623455dc72 service neutron] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:45:10.331992 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4575f564-5a29-4aa8-85cd-61623455dc72 service neutron] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:45:10.332369 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4575f564-5a29-4aa8-85cd-61623455dc72 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:45:10.342468 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4575f564-5a29-4aa8-85cd-61623455dc72 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:45:10.344622 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4575f564-5a29-4aa8-85cd-61623455dc72 service neutron] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:45:10.357298 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1362/2730] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jul 25 05:45:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 05:45:10.860958 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90eb812f-0917-42f6-bdcf-79220899b587 None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:45:10.863106 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-90eb812f-0917-42f6-bdcf-79220899b587 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:45:10.882822 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90eb812f-0917-42f6-bdcf-79220899b587 None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:45:10.894354 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff39d3d3-5820-461e-9283-3f3d46879254 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:45:10.895460 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff39d3d3-5820-461e-9283-3f3d46879254 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:45:10.895915 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff39d3d3-5820-461e-9283-3f3d46879254 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:45:10.896290 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff39d3d3-5820-461e-9283-3f3d46879254 admin admin] PATH_INFO: `/v3/projects/4630ba0434314737a8b673a7288646f7` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:45:10.898304 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff39d3d3-5820-461e-9283-3f3d46879254 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4630ba0434314737a8b673a7288646f7)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:45:10.901635 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff39d3d3-5820-461e-9283-3f3d46879254 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:45:11.017278 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-ff39d3d3-5820-461e-9283-3f3d46879254 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4630ba0434314737a8b673a7288646f7', 'request_id': 'req-ff39d3d3-5820-461e-9283-3f3d46879254'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:45:11.026563 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-ff39d3d3-5820-461e-9283-3f3d46879254 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4630ba0434314737a8b673a7288646f7', 'request_id': 'req-ff39d3d3-5820-461e-9283-3f3d46879254'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:45:11.039605 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-ff39d3d3-5820-461e-9283-3f3d46879254 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4630ba0434314737a8b673a7288646f7', 'request_id': 'req-ff39d3d3-5820-461e-9283-3f3d46879254'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:45:11.069158 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-ff39d3d3-5820-461e-9283-3f3d46879254 admin admin] The token cache is being invalidate because project 4630ba0434314737a8b673a7288646f7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80248) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 05:45:11.069714 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-ff39d3d3-5820-461e-9283-3f3d46879254 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:45:11.070660 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-ff39d3d3-5820-461e-9283-3f3d46879254 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:45:11.073129 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1369/2731] 10.222.0.33 () {58 vars in 1232 bytes} [Mon Jul 25 05:45:10 2022] DELETE /identity/v3/projects/4630ba0434314737a8b673a7288646f7 => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 05:45:12.489136 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4575f564-5a29-4aa8-85cd-61623455dc72 service neutron] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:45:12.490659 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-4575f564-5a29-4aa8-85cd-61623455dc72 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:45:12.712815 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4575f564-5a29-4aa8-85cd-61623455dc72 service neutron] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:45:12.740370 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:45:12.813209 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d59f364-e2d5-4d14-9064-ac56de5cbe46 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '317790016e7047fe96af73925de5d872', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92afba266cce4d1bad4134c351e74be3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:45:12.813869 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d59f364-e2d5-4d14-9064-ac56de5cbe46 service placement] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:45:12.814152 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d59f364-e2d5-4d14-9064-ac56de5cbe46 service placement] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:45:12.814431 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d59f364-e2d5-4d14-9064-ac56de5cbe46 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:45:12.817019 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-9d59f364-e2d5-4d14-9064-ac56de5cbe46 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:45:12.841476 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d59f364-e2d5-4d14-9064-ac56de5cbe46 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:45:12.843081 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d59f364-e2d5-4d14-9064-ac56de5cbe46 service placement] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:45:12.854094 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1363/2732] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jul 25 05:45:12 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 366 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 05:45:13.734044 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff39d3d3-5820-461e-9283-3f3d46879254 admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:45:13.735796 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-ff39d3d3-5820-461e-9283-3f3d46879254 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:45:13.801108 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff39d3d3-5820-461e-9283-3f3d46879254 admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:45:13.804727 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7dd8b55-43b5-44b1-9bf2-70dd022eabd8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db57b478c25643dbaf936f90f6e17d0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92afba266cce4d1bad4134c351e74be3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:45:13.805407 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7dd8b55-43b5-44b1-9bf2-70dd022eabd8 service cinder] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:45:13.805842 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7dd8b55-43b5-44b1-9bf2-70dd022eabd8 service cinder] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:45:13.806168 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7dd8b55-43b5-44b1-9bf2-70dd022eabd8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:45:13.809312 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-e7dd8b55-43b5-44b1-9bf2-70dd022eabd8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:45:13.882678 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7dd8b55-43b5-44b1-9bf2-70dd022eabd8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:45:13.884830 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7dd8b55-43b5-44b1-9bf2-70dd022eabd8 service cinder] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:45:13.898957 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1370/2733] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jul 25 05:45:13 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 165 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 05:45:14.050072 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c3ad1b-0888-4896-a686-86b5194c9e60 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:45:14.050552 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c3ad1b-0888-4896-a686-86b5194c9e60 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:45:14.050654 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c3ad1b-0888-4896-a686-86b5194c9e60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:45:14.071255 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-a0c3ad1b-0888-4896-a686-86b5194c9e60 None None] MFA Rules not processed for user `3352531be1b54f76b864383ccc2e5ec2`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:45:14.072578 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-a0c3ad1b-0888-4896-a686-86b5194c9e60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:45:14.144238 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1364/2734] 10.222.0.33 () {58 vars in 967 bytes} [Mon Jul 25 05:45:14 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:45:14.159693 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7dd8b55-43b5-44b1-9bf2-70dd022eabd8 service cinder] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:45:14.161259 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-e7dd8b55-43b5-44b1-9bf2-70dd022eabd8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:45:14.213007 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:45:14.294800 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7dd8b55-43b5-44b1-9bf2-70dd022eabd8 service cinder] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:45:14.318131 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-101d647c-77be-4a1c-bc5f-5b3a61c99f00 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb135a8d5d7444cfbfa336115ba2ba44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92afba266cce4d1bad4134c351e74be3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:45:14.319087 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101d647c-77be-4a1c-bc5f-5b3a61c99f00 service swift] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:45:14.319541 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101d647c-77be-4a1c-bc5f-5b3a61c99f00 service swift] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:45:14.320008 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101d647c-77be-4a1c-bc5f-5b3a61c99f00 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:45:14.341250 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-101d647c-77be-4a1c-bc5f-5b3a61c99f00 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:45:14.343697 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-101d647c-77be-4a1c-bc5f-5b3a61c99f00 service swift] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:45:14.357938 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1371/2735] 10.222.0.33 () {60 vars in 1384 bytes} [Mon Jul 25 05:45:14 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 198 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 25 05:45:17.366870 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c3ad1b-0888-4896-a686-86b5194c9e60 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:45:17.370228 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-a0c3ad1b-0888-4896-a686-86b5194c9e60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:45:17.441601 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c3ad1b-0888-4896-a686-86b5194c9e60 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:45:17.446402 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9594aa47-1f92-417f-9df0-4c425bcc333d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:45:17.447332 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9594aa47-1f92-417f-9df0-4c425bcc333d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:45:17.447618 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9594aa47-1f92-417f-9df0-4c425bcc333d admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:45:17.447899 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9594aa47-1f92-417f-9df0-4c425bcc333d admin admin] PATH_INFO: `/v3/users/598f7eff11f545a88d339812207f0f19` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:45:17.449187 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9594aa47-1f92-417f-9df0-4c425bcc333d admin admin] RBAC: Authorizing `identity:delete_user(user_id=598f7eff11f545a88d339812207f0f19)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:45:17.451129 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9594aa47-1f92-417f-9df0-4c425bcc333d admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:45:17.503497 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-9594aa47-1f92-417f-9df0-4c425bcc333d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '598f7eff11f545a88d339812207f0f19'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:45:17.504082 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-9594aa47-1f92-417f-9df0-4c425bcc333d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '598f7eff11f545a88d339812207f0f19'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:45:17.508720 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:45:17.512173 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-9594aa47-1f92-417f-9df0-4c425bcc333d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '598f7eff11f545a88d339812207f0f19'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:45:17.526156 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-9594aa47-1f92-417f-9df0-4c425bcc333d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '598f7eff11f545a88d339812207f0f19'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:45:17.526897 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-9594aa47-1f92-417f-9df0-4c425bcc333d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '598f7eff11f545a88d339812207f0f19'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:45:17.546602 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1365/2736] 10.222.0.33 () {58 vars in 1223 bytes} [Mon Jul 25 05:45:17 2022] DELETE /identity/v3/users/598f7eff11f545a88d339812207f0f19 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 05:45:17.834957 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-101d647c-77be-4a1c-bc5f-5b3a61c99f00 service swift] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:45:17.847253 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-101d647c-77be-4a1c-bc5f-5b3a61c99f00 service swift] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:45:17.911350 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ea9c45-aca7-4524-891b-15435e0ac157 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:45:17.912415 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ea9c45-aca7-4524-891b-15435e0ac157 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:45:17.912833 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ea9c45-aca7-4524-891b-15435e0ac157 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:45:17.913219 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ea9c45-aca7-4524-891b-15435e0ac157 admin admin] PATH_INFO: `/v3/projects/987ce20ceedd49cb9419cc506c50fce3` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:45:17.914553 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16ea9c45-aca7-4524-891b-15435e0ac157 admin admin] RBAC: Authorizing `identity:delete_project(project_id=987ce20ceedd49cb9419cc506c50fce3)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:45:17.917263 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16ea9c45-aca7-4524-891b-15435e0ac157 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:45:17.967929 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-16ea9c45-aca7-4524-891b-15435e0ac157 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '987ce20ceedd49cb9419cc506c50fce3', 'request_id': 'req-16ea9c45-aca7-4524-891b-15435e0ac157'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:45:17.973779 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-16ea9c45-aca7-4524-891b-15435e0ac157 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '987ce20ceedd49cb9419cc506c50fce3', 'request_id': 'req-16ea9c45-aca7-4524-891b-15435e0ac157'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:45:17.981222 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-16ea9c45-aca7-4524-891b-15435e0ac157 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '987ce20ceedd49cb9419cc506c50fce3', 'request_id': 'req-16ea9c45-aca7-4524-891b-15435e0ac157'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:45:17.994248 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-16ea9c45-aca7-4524-891b-15435e0ac157 admin admin] The token cache is being invalidate because project 987ce20ceedd49cb9419cc506c50fce3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80248) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 05:45:17.994248 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-16ea9c45-aca7-4524-891b-15435e0ac157 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:45:17.994873 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-16ea9c45-aca7-4524-891b-15435e0ac157 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:45:17.996478 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1372/2737] 10.222.0.33 () {58 vars in 1232 bytes} [Mon Jul 25 05:45:17 2022] DELETE /identity/v3/projects/987ce20ceedd49cb9419cc506c50fce3 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 05:45:18.000506 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9594aa47-1f92-417f-9df0-4c425bcc333d admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:45:18.001942 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-9594aa47-1f92-417f-9df0-4c425bcc333d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:45:18.017686 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9594aa47-1f92-417f-9df0-4c425bcc333d admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:45:18.081225 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7dd4b49-5d08-4c2d-b320-9f1e723b23c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:45:18.082137 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7dd4b49-5d08-4c2d-b320-9f1e723b23c3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:45:18.082423 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7dd4b49-5d08-4c2d-b320-9f1e723b23c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:45:18.082705 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7dd4b49-5d08-4c2d-b320-9f1e723b23c3 admin admin] PATH_INFO: `/v3/users/6b09e9366f36460fbb934b7b3fe881a5` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:45:18.083938 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7dd4b49-5d08-4c2d-b320-9f1e723b23c3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6b09e9366f36460fbb934b7b3fe881a5)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:45:18.085929 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7dd4b49-5d08-4c2d-b320-9f1e723b23c3 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:45:18.145511 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-b7dd4b49-5d08-4c2d-b320-9f1e723b23c3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6b09e9366f36460fbb934b7b3fe881a5'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:45:18.146240 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-b7dd4b49-5d08-4c2d-b320-9f1e723b23c3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6b09e9366f36460fbb934b7b3fe881a5'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:45:18.158713 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-b7dd4b49-5d08-4c2d-b320-9f1e723b23c3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6b09e9366f36460fbb934b7b3fe881a5'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:45:18.180276 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-b7dd4b49-5d08-4c2d-b320-9f1e723b23c3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6b09e9366f36460fbb934b7b3fe881a5'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:45:18.181734 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-b7dd4b49-5d08-4c2d-b320-9f1e723b23c3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6b09e9366f36460fbb934b7b3fe881a5'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:45:18.195093 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1366/2738] 10.222.0.33 () {58 vars in 1223 bytes} [Mon Jul 25 05:45:17 2022] DELETE /identity/v3/users/6b09e9366f36460fbb934b7b3fe881a5 => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 05:45:18.371509 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ea9c45-aca7-4524-891b-15435e0ac157 admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:45:18.373713 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-16ea9c45-aca7-4524-891b-15435e0ac157 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:45:18.387753 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ea9c45-aca7-4524-891b-15435e0ac157 admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:45:18.432355 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:45:18.452044 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb573592-ca1f-495a-8293-0b589bdbfe4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:45:18.452959 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb573592-ca1f-495a-8293-0b589bdbfe4b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:45:18.453249 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb573592-ca1f-495a-8293-0b589bdbfe4b admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:45:18.453586 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb573592-ca1f-495a-8293-0b589bdbfe4b admin admin] PATH_INFO: `/v3/projects/968e41b62cc34e7198a1e55dd1a7d858` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:45:18.454790 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb573592-ca1f-495a-8293-0b589bdbfe4b admin admin] RBAC: Authorizing `identity:delete_project(project_id=968e41b62cc34e7198a1e55dd1a7d858)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:45:18.456696 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb573592-ca1f-495a-8293-0b589bdbfe4b admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:45:18.506581 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-eb573592-ca1f-495a-8293-0b589bdbfe4b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '968e41b62cc34e7198a1e55dd1a7d858', 'request_id': 'req-eb573592-ca1f-495a-8293-0b589bdbfe4b'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:45:18.512099 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-eb573592-ca1f-495a-8293-0b589bdbfe4b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '968e41b62cc34e7198a1e55dd1a7d858', 'request_id': 'req-eb573592-ca1f-495a-8293-0b589bdbfe4b'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:45:18.518260 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-eb573592-ca1f-495a-8293-0b589bdbfe4b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '968e41b62cc34e7198a1e55dd1a7d858', 'request_id': 'req-eb573592-ca1f-495a-8293-0b589bdbfe4b'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:45:18.531446 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-eb573592-ca1f-495a-8293-0b589bdbfe4b admin admin] The token cache is being invalidate because project 968e41b62cc34e7198a1e55dd1a7d858 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80248) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 05:45:18.531734 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-eb573592-ca1f-495a-8293-0b589bdbfe4b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:45:18.532380 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-eb573592-ca1f-495a-8293-0b589bdbfe4b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:45:18.533952 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1373/2739] 10.222.0.33 () {58 vars in 1232 bytes} [Mon Jul 25 05:45:18 2022] DELETE /identity/v3/projects/968e41b62cc34e7198a1e55dd1a7d858 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 05:45:19.201985 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7dd4b49-5d08-4c2d-b320-9f1e723b23c3 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:45:19.203411 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-b7dd4b49-5d08-4c2d-b320-9f1e723b23c3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:45:19.216474 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7dd4b49-5d08-4c2d-b320-9f1e723b23c3 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:45:19.260772 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:45:19.274019 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-914c0126-4803-4d24-aa1b-1aaba1a74cfb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:45:19.275430 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914c0126-4803-4d24-aa1b-1aaba1a74cfb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:45:19.275712 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914c0126-4803-4d24-aa1b-1aaba1a74cfb admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:45:19.275986 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914c0126-4803-4d24-aa1b-1aaba1a74cfb admin admin] PATH_INFO: `/v3/users/5b0f119c38564a499386edbd3673c5b6` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:45:19.277195 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-914c0126-4803-4d24-aa1b-1aaba1a74cfb admin admin] RBAC: Authorizing `identity:delete_user(user_id=5b0f119c38564a499386edbd3673c5b6)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:45:19.279088 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-914c0126-4803-4d24-aa1b-1aaba1a74cfb admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:45:19.323053 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-914c0126-4803-4d24-aa1b-1aaba1a74cfb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5b0f119c38564a499386edbd3673c5b6'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:45:19.323698 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-914c0126-4803-4d24-aa1b-1aaba1a74cfb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5b0f119c38564a499386edbd3673c5b6'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:45:19.330524 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-914c0126-4803-4d24-aa1b-1aaba1a74cfb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5b0f119c38564a499386edbd3673c5b6'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:45:19.343477 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-914c0126-4803-4d24-aa1b-1aaba1a74cfb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5b0f119c38564a499386edbd3673c5b6'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:45:19.344174 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-914c0126-4803-4d24-aa1b-1aaba1a74cfb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5b0f119c38564a499386edbd3673c5b6'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:45:19.357661 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1367/2740] 10.222.0.33 () {58 vars in 1223 bytes} [Mon Jul 25 05:45:19 2022] DELETE /identity/v3/users/5b0f119c38564a499386edbd3673c5b6 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 05:45:19.362163 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10919d9f-7ee0-4372-a52a-f7f52f14b5b7 None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:45:19.362658 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10919d9f-7ee0-4372-a52a-f7f52f14b5b7 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:45:19.362788 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10919d9f-7ee0-4372-a52a-f7f52f14b5b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:45:19.380540 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-10919d9f-7ee0-4372-a52a-f7f52f14b5b7 None None] MFA Rules not processed for user `1eae540242d94c10b0ad33960a855f08`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:45:19.381773 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-10919d9f-7ee0-4372-a52a-f7f52f14b5b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:45:19.440741 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1374/2741] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 25 05:45:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:45:19.450603 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-914c0126-4803-4d24-aa1b-1aaba1a74cfb admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:45:19.452013 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-914c0126-4803-4d24-aa1b-1aaba1a74cfb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:45:19.509104 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-914c0126-4803-4d24-aa1b-1aaba1a74cfb admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:45:19.513833 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce832926-06ca-4373-bcef-98022ba01dfe service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b2931200f99e43d79a50a9f6e02738e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92afba266cce4d1bad4134c351e74be3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:45:19.514481 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce832926-06ca-4373-bcef-98022ba01dfe service neutron] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:45:19.514767 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce832926-06ca-4373-bcef-98022ba01dfe service neutron] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:45:19.515044 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce832926-06ca-4373-bcef-98022ba01dfe service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:45:19.523984 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce832926-06ca-4373-bcef-98022ba01dfe service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:45:19.526338 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce832926-06ca-4373-bcef-98022ba01dfe service neutron] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:45:19.535682 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1368/2742] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jul 25 05:45:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 05:45:19.805170 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10919d9f-7ee0-4372-a52a-f7f52f14b5b7 None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:45:19.806612 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-10919d9f-7ee0-4372-a52a-f7f52f14b5b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:45:19.820606 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10919d9f-7ee0-4372-a52a-f7f52f14b5b7 None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:45:19.826196 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd766d10-1127-48d5-94b1-46e07dc3b16a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:45:19.827623 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd766d10-1127-48d5-94b1-46e07dc3b16a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:45:19.827910 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd766d10-1127-48d5-94b1-46e07dc3b16a admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:45:19.828362 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd766d10-1127-48d5-94b1-46e07dc3b16a admin admin] PATH_INFO: `/v3/projects/9e542cc2bf96413a85d2eeb1776145ab` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:45:19.829900 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd766d10-1127-48d5-94b1-46e07dc3b16a admin admin] RBAC: Authorizing `identity:delete_project(project_id=9e542cc2bf96413a85d2eeb1776145ab)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:45:19.832266 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd766d10-1127-48d5-94b1-46e07dc3b16a admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:45:19.879806 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-bd766d10-1127-48d5-94b1-46e07dc3b16a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9e542cc2bf96413a85d2eeb1776145ab', 'request_id': 'req-bd766d10-1127-48d5-94b1-46e07dc3b16a'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:45:19.886120 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-bd766d10-1127-48d5-94b1-46e07dc3b16a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9e542cc2bf96413a85d2eeb1776145ab', 'request_id': 'req-bd766d10-1127-48d5-94b1-46e07dc3b16a'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:45:19.892349 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-bd766d10-1127-48d5-94b1-46e07dc3b16a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9e542cc2bf96413a85d2eeb1776145ab', 'request_id': 'req-bd766d10-1127-48d5-94b1-46e07dc3b16a'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:45:19.895927 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:45:19.905867 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-bd766d10-1127-48d5-94b1-46e07dc3b16a admin admin] The token cache is being invalidate because project 9e542cc2bf96413a85d2eeb1776145ab was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80248) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 05:45:19.906383 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-bd766d10-1127-48d5-94b1-46e07dc3b16a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:45:19.907136 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-bd766d10-1127-48d5-94b1-46e07dc3b16a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:45:19.908844 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1375/2743] 10.222.0.33 () {58 vars in 1232 bytes} [Mon Jul 25 05:45:19 2022] DELETE /identity/v3/projects/9e542cc2bf96413a85d2eeb1776145ab => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 05:45:19.912751 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce832926-06ca-4373-bcef-98022ba01dfe service neutron] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:45:19.922521 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce832926-06ca-4373-bcef-98022ba01dfe service neutron] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:45:19.979942 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f806575-2485-4e85-aa2a-406e0518cc2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:45:19.980743 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f806575-2485-4e85-aa2a-406e0518cc2c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:45:19.981064 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f806575-2485-4e85-aa2a-406e0518cc2c admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:45:19.981339 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f806575-2485-4e85-aa2a-406e0518cc2c admin admin] PATH_INFO: `/v3/users/6e7dd002675043a8a384e2478ae0ca2c` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:45:19.982570 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f806575-2485-4e85-aa2a-406e0518cc2c admin admin] RBAC: Authorizing `identity:delete_user(user_id=6e7dd002675043a8a384e2478ae0ca2c)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:45:19.984482 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f806575-2485-4e85-aa2a-406e0518cc2c admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:45:19.997764 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:45:20.035715 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-7f806575-2485-4e85-aa2a-406e0518cc2c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6e7dd002675043a8a384e2478ae0ca2c'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:45:20.036634 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-7f806575-2485-4e85-aa2a-406e0518cc2c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6e7dd002675043a8a384e2478ae0ca2c'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:45:20.043994 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-7f806575-2485-4e85-aa2a-406e0518cc2c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6e7dd002675043a8a384e2478ae0ca2c'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:45:20.056028 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-7f806575-2485-4e85-aa2a-406e0518cc2c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6e7dd002675043a8a384e2478ae0ca2c'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:45:20.057797 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-7f806575-2485-4e85-aa2a-406e0518cc2c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6e7dd002675043a8a384e2478ae0ca2c'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:45:20.073859 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1369/2744] 10.222.0.33 () {58 vars in 1223 bytes} [Mon Jul 25 05:45:19 2022] DELETE /identity/v3/users/6e7dd002675043a8a384e2478ae0ca2c => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 05:45:20.593598 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd766d10-1127-48d5-94b1-46e07dc3b16a admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:45:20.595123 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-bd766d10-1127-48d5-94b1-46e07dc3b16a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:45:20.609328 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd766d10-1127-48d5-94b1-46e07dc3b16a admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:45:20.669442 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb5905f-66a4-4715-91e2-42fc88da5207 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:45:20.670296 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb5905f-66a4-4715-91e2-42fc88da5207 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:45:20.670599 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb5905f-66a4-4715-91e2-42fc88da5207 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:45:20.670862 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb5905f-66a4-4715-91e2-42fc88da5207 admin admin] PATH_INFO: `/v3/projects/bf1aa3e44dc54046853fe847d0b37325` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:45:20.672042 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdb5905f-66a4-4715-91e2-42fc88da5207 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bf1aa3e44dc54046853fe847d0b37325)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:45:20.673969 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdb5905f-66a4-4715-91e2-42fc88da5207 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:45:20.719879 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-cdb5905f-66a4-4715-91e2-42fc88da5207 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bf1aa3e44dc54046853fe847d0b37325', 'request_id': 'req-cdb5905f-66a4-4715-91e2-42fc88da5207'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:45:20.725357 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-cdb5905f-66a4-4715-91e2-42fc88da5207 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bf1aa3e44dc54046853fe847d0b37325', 'request_id': 'req-cdb5905f-66a4-4715-91e2-42fc88da5207'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:45:20.732121 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-cdb5905f-66a4-4715-91e2-42fc88da5207 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bf1aa3e44dc54046853fe847d0b37325', 'request_id': 'req-cdb5905f-66a4-4715-91e2-42fc88da5207'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:45:20.745304 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-cdb5905f-66a4-4715-91e2-42fc88da5207 admin admin] The token cache is being invalidate because project bf1aa3e44dc54046853fe847d0b37325 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80248) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 05:45:20.745597 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-cdb5905f-66a4-4715-91e2-42fc88da5207 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:45:20.746239 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-cdb5905f-66a4-4715-91e2-42fc88da5207 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:45:20.747785 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1376/2745] 10.222.0.33 () {58 vars in 1232 bytes} [Mon Jul 25 05:45:20 2022] DELETE /identity/v3/projects/bf1aa3e44dc54046853fe847d0b37325 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 05:45:20.753626 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b61d359-6fbf-47f6-8839-327566ff45ad None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:45:20.754091 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b61d359-6fbf-47f6-8839-327566ff45ad None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:45:20.754232 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b61d359-6fbf-47f6-8839-327566ff45ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:45:20.772642 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-4b61d359-6fbf-47f6-8839-327566ff45ad None None] MFA Rules not processed for user `1eae540242d94c10b0ad33960a855f08`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:45:20.773840 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-4b61d359-6fbf-47f6-8839-327566ff45ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:45:20.833059 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1370/2746] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 25 05:45:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:45:20.894896 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad90c77d-c63b-4442-8a6e-f98c8ac3ec6e None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:45:20.895184 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad90c77d-c63b-4442-8a6e-f98c8ac3ec6e None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:45:20.895463 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad90c77d-c63b-4442-8a6e-f98c8ac3ec6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:45:20.914412 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-ad90c77d-c63b-4442-8a6e-f98c8ac3ec6e None None] MFA Rules not processed for user `1eae540242d94c10b0ad33960a855f08`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:45:20.915759 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-ad90c77d-c63b-4442-8a6e-f98c8ac3ec6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:45:20.923550 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1377/2747] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 25 05:45:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:45:20.927825 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b61d359-6fbf-47f6-8839-327566ff45ad None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:45:20.938202 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b61d359-6fbf-47f6-8839-327566ff45ad None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:45:20.942838 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4002a5c9-73e0-4127-98e8-d4ee07cfe9ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:45:20.943492 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4002a5c9-73e0-4127-98e8-d4ee07cfe9ba admin admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:45:20.943776 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4002a5c9-73e0-4127-98e8-d4ee07cfe9ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:45:20.944052 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4002a5c9-73e0-4127-98e8-d4ee07cfe9ba admin admin] PATH_INFO: `/v3/domains` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:45:20.944806 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4002a5c9-73e0-4127-98e8-d4ee07cfe9ba admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:45:20.945125 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4002a5c9-73e0-4127-98e8-d4ee07cfe9ba admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:45:20.946709 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4002a5c9-73e0-4127-98e8-d4ee07cfe9ba admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:45:20.958808 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1371/2748] 10.222.0.33 () {58 vars in 1152 bytes} [Mon Jul 25 05:45:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:45:20.963369 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad90c77d-c63b-4442-8a6e-f98c8ac3ec6e None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:45:20.972511 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad90c77d-c63b-4442-8a6e-f98c8ac3ec6e None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:45:20.977472 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b2bab43-2f1b-44b0-841f-bff3028de395 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:45:20.979017 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2bab43-2f1b-44b0-841f-bff3028de395 admin admin] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:45:20.979398 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2bab43-2f1b-44b0-841f-bff3028de395 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:45:20.979686 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2bab43-2f1b-44b0-841f-bff3028de395 admin admin] PATH_INFO: `/v3/projects` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:45:20.980384 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b2bab43-2f1b-44b0-841f-bff3028de395 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:45:20.982350 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b2bab43-2f1b-44b0-841f-bff3028de395 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:45:21.007691 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1378/2749] 10.222.0.33 () {60 vars in 1152 bytes} [Mon Jul 25 05:45:20 2022] POST /identity/v3/projects => generated 371 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 05:45:21.011648 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4002a5c9-73e0-4127-98e8-d4ee07cfe9ba admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:45:21.021663 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4002a5c9-73e0-4127-98e8-d4ee07cfe9ba admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:45:21.077312 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2713ca99-65ce-4ab6-a82d-c1b632e79b4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:45:21.077967 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2713ca99-65ce-4ab6-a82d-c1b632e79b4b admin admin] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:45:21.078259 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2713ca99-65ce-4ab6-a82d-c1b632e79b4b admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:45:21.078541 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2713ca99-65ce-4ab6-a82d-c1b632e79b4b admin admin] PATH_INFO: `/v3/users` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:45:21.079229 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2713ca99-65ce-4ab6-a82d-c1b632e79b4b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:45:21.081171 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2713ca99-65ce-4ab6-a82d-c1b632e79b4b admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:45:21.100980 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1372/2750] 10.222.0.33 () {60 vars in 1143 bytes} [Mon Jul 25 05:45:21 2022] POST /identity/v3/users => generated 286 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 25 05:45:21.105401 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b2bab43-2f1b-44b0-841f-bff3028de395 admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:45:21.115750 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b2bab43-2f1b-44b0-841f-bff3028de395 admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:45:21.120386 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44911b46-edf1-4c49-95c8-08e2e36a4ac6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:45:21.121082 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44911b46-edf1-4c49-95c8-08e2e36a4ac6 admin admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:45:21.121375 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44911b46-edf1-4c49-95c8-08e2e36a4ac6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:45:21.121657 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44911b46-edf1-4c49-95c8-08e2e36a4ac6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:45:21.122420 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44911b46-edf1-4c49-95c8-08e2e36a4ac6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:45:21.124118 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44911b46-edf1-4c49-95c8-08e2e36a4ac6 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:45:21.138142 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1379/2751] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jul 25 05:45:21 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 05:45:21.139580 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2713ca99-65ce-4ab6-a82d-c1b632e79b4b admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:45:21.149539 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2713ca99-65ce-4ab6-a82d-c1b632e79b4b admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:45:21.154921 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be74c01e-9150-476f-9976-654ae1b7cd1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:45:21.155672 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be74c01e-9150-476f-9976-654ae1b7cd1c admin admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:45:21.156017 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be74c01e-9150-476f-9976-654ae1b7cd1c admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:45:21.156360 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be74c01e-9150-476f-9976-654ae1b7cd1c admin admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:45:21.157199 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be74c01e-9150-476f-9976-654ae1b7cd1c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:45:21.158853 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be74c01e-9150-476f-9976-654ae1b7cd1c admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:45:21.160259 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=87,keystone:DELETE=9,keystone:INSERT=4 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:45:21.172000 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1373/2752] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jul 25 05:45:21 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 05:45:21.176118 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44911b46-edf1-4c49-95c8-08e2e36a4ac6 admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:45:21.186748 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44911b46-edf1-4c49-95c8-08e2e36a4ac6 admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:45:21.191794 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec26840b-710a-4352-bda2-1cde8abd6119 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:45:21.192689 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec26840b-710a-4352-bda2-1cde8abd6119 admin admin] REQUEST_METHOD: `PUT` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:45:21.193002 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec26840b-710a-4352-bda2-1cde8abd6119 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:45:21.193339 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec26840b-710a-4352-bda2-1cde8abd6119 admin admin] PATH_INFO: `/v3/projects/8915bf8a11f64a2ca20675d67753dc45/users/59b24cf4ffc240709e8d3c005bc22633/roles/5074e74e35f54d4b9bb415c34999a82a` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:45:21.209516 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec26840b-710a-4352-bda2-1cde8abd6119 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8915bf8a11f64a2ca20675d67753dc45, user_id=59b24cf4ffc240709e8d3c005bc22633, role_id=5074e74e35f54d4b9bb415c34999a82a)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:45:21.212319 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec26840b-710a-4352-bda2-1cde8abd6119 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:45:21.227090 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1380/2753] 10.222.0.33 () {60 vars in 1482 bytes} [Mon Jul 25 05:45:21 2022] PUT /identity/v3/projects/8915bf8a11f64a2ca20675d67753dc45/users/59b24cf4ffc240709e8d3c005bc22633/roles/5074e74e35f54d4b9bb415c34999a82a => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 05:45:21.237975 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde37eb4-f440-469d-958c-6071ef7ac460 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:45:21.238351 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde37eb4-f440-469d-958c-6071ef7ac460 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:45:21.238731 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde37eb4-f440-469d-958c-6071ef7ac460 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:45:21.266254 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-bde37eb4-f440-469d-958c-6071ef7ac460 None None] MFA Rules not processed for user `59b24cf4ffc240709e8d3c005bc22633`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:45:21.267480 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-bde37eb4-f440-469d-958c-6071ef7ac460 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:45:21.331411 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1374/2754] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 25 05:45:21 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 05:45:21.343279 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec26840b-710a-4352-bda2-1cde8abd6119 admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:45:21.345013 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-ec26840b-710a-4352-bda2-1cde8abd6119 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:45:21.372754 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:45:21.405371 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec26840b-710a-4352-bda2-1cde8abd6119 admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:45:21.409035 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13daeeb6-d97a-49c0-9749-f0c707957bd2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db57b478c25643dbaf936f90f6e17d0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92afba266cce4d1bad4134c351e74be3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:45:21.409678 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13daeeb6-d97a-49c0-9749-f0c707957bd2 service cinder] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:45:21.409966 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13daeeb6-d97a-49c0-9749-f0c707957bd2 service cinder] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:45:21.410243 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13daeeb6-d97a-49c0-9749-f0c707957bd2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:45:21.419031 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13daeeb6-d97a-49c0-9749-f0c707957bd2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:45:21.420451 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13daeeb6-d97a-49c0-9749-f0c707957bd2 service cinder] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:45:21.429158 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1381/2755] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jul 25 05:45:21 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 05:45:21.448436 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde37eb4-f440-469d-958c-6071ef7ac460 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:45:21.510952 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde37eb4-f440-469d-958c-6071ef7ac460 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:45:21.515713 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de947742-d66d-43e8-9e92-68c1ec31bf3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:45:21.516579 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de947742-d66d-43e8-9e92-68c1ec31bf3b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:45:21.516898 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de947742-d66d-43e8-9e92-68c1ec31bf3b admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:45:21.517177 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de947742-d66d-43e8-9e92-68c1ec31bf3b admin admin] PATH_INFO: `/v3/users/59b24cf4ffc240709e8d3c005bc22633` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:45:21.518381 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de947742-d66d-43e8-9e92-68c1ec31bf3b admin admin] RBAC: Authorizing `identity:delete_user(user_id=59b24cf4ffc240709e8d3c005bc22633)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:45:21.520296 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de947742-d66d-43e8-9e92-68c1ec31bf3b admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:45:21.563807 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-de947742-d66d-43e8-9e92-68c1ec31bf3b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '59b24cf4ffc240709e8d3c005bc22633'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:45:21.564490 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-de947742-d66d-43e8-9e92-68c1ec31bf3b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '59b24cf4ffc240709e8d3c005bc22633'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:45:21.571448 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-de947742-d66d-43e8-9e92-68c1ec31bf3b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '59b24cf4ffc240709e8d3c005bc22633'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:45:21.583641 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-de947742-d66d-43e8-9e92-68c1ec31bf3b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '59b24cf4ffc240709e8d3c005bc22633'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:45:21.584246 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-de947742-d66d-43e8-9e92-68c1ec31bf3b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '59b24cf4ffc240709e8d3c005bc22633'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:45:21.597241 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1375/2756] 10.222.0.33 () {58 vars in 1223 bytes} [Mon Jul 25 05:45:21 2022] DELETE /identity/v3/users/59b24cf4ffc240709e8d3c005bc22633 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 05:45:21.601975 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b10d34-3b54-4a46-a569-2421cff773ac None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:45:21.602432 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b10d34-3b54-4a46-a569-2421cff773ac None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:45:21.602532 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b10d34-3b54-4a46-a569-2421cff773ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:45:21.621792 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-95b10d34-3b54-4a46-a569-2421cff773ac None None] MFA Rules not processed for user `1eae540242d94c10b0ad33960a855f08`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:45:21.623027 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-95b10d34-3b54-4a46-a569-2421cff773ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:45:21.684385 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1382/2757] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 25 05:45:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:45:21.695851 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de947742-d66d-43e8-9e92-68c1ec31bf3b admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:45:21.698244 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-de947742-d66d-43e8-9e92-68c1ec31bf3b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:45:21.705106 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:45:21.758150 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de947742-d66d-43e8-9e92-68c1ec31bf3b admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:45:21.762006 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca530e8-a8a7-4d0c-8868-b7bf7d7c415e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b2931200f99e43d79a50a9f6e02738e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92afba266cce4d1bad4134c351e74be3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:45:21.762655 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca530e8-a8a7-4d0c-8868-b7bf7d7c415e service neutron] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:45:21.762941 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca530e8-a8a7-4d0c-8868-b7bf7d7c415e service neutron] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:45:21.763220 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca530e8-a8a7-4d0c-8868-b7bf7d7c415e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:45:21.772248 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ca530e8-a8a7-4d0c-8868-b7bf7d7c415e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:45:21.773712 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ca530e8-a8a7-4d0c-8868-b7bf7d7c415e service neutron] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:45:21.783443 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1376/2758] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jul 25 05:45:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 05:45:22.056603 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b10d34-3b54-4a46-a569-2421cff773ac None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:45:22.058090 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-95b10d34-3b54-4a46-a569-2421cff773ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:45:22.071709 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b10d34-3b54-4a46-a569-2421cff773ac None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:45:22.077961 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15abf5f6-78e7-418d-ae19-9062cdd459c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:45:22.078968 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15abf5f6-78e7-418d-ae19-9062cdd459c9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:45:22.079261 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15abf5f6-78e7-418d-ae19-9062cdd459c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:45:22.079541 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15abf5f6-78e7-418d-ae19-9062cdd459c9 admin admin] PATH_INFO: `/v3/projects/8915bf8a11f64a2ca20675d67753dc45` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:45:22.080764 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15abf5f6-78e7-418d-ae19-9062cdd459c9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8915bf8a11f64a2ca20675d67753dc45)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:45:22.082706 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15abf5f6-78e7-418d-ae19-9062cdd459c9 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:45:22.131053 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-15abf5f6-78e7-418d-ae19-9062cdd459c9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8915bf8a11f64a2ca20675d67753dc45', 'request_id': 'req-15abf5f6-78e7-418d-ae19-9062cdd459c9'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:45:22.137614 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-15abf5f6-78e7-418d-ae19-9062cdd459c9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8915bf8a11f64a2ca20675d67753dc45', 'request_id': 'req-15abf5f6-78e7-418d-ae19-9062cdd459c9'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:45:22.143899 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-15abf5f6-78e7-418d-ae19-9062cdd459c9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8915bf8a11f64a2ca20675d67753dc45', 'request_id': 'req-15abf5f6-78e7-418d-ae19-9062cdd459c9'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:45:22.155867 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-15abf5f6-78e7-418d-ae19-9062cdd459c9 admin admin] The token cache is being invalidate because project 8915bf8a11f64a2ca20675d67753dc45 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80248) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 05:45:22.156154 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-15abf5f6-78e7-418d-ae19-9062cdd459c9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:45:22.156767 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-15abf5f6-78e7-418d-ae19-9062cdd459c9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:45:22.158324 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1383/2759] 10.222.0.33 () {58 vars in 1232 bytes} [Mon Jul 25 05:45:22 2022] DELETE /identity/v3/projects/8915bf8a11f64a2ca20675d67753dc45 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 05:45:22.165267 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1b5b2d-e0d7-4cef-a8cf-3653a0ac73d0 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:45:22.165584 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1b5b2d-e0d7-4cef-a8cf-3653a0ac73d0 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:45:22.165893 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1b5b2d-e0d7-4cef-a8cf-3653a0ac73d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:45:22.184319 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-da1b5b2d-e0d7-4cef-a8cf-3653a0ac73d0 None None] MFA Rules not processed for user `1eae540242d94c10b0ad33960a855f08`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:45:22.185636 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-da1b5b2d-e0d7-4cef-a8cf-3653a0ac73d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:45:22.244947 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1377/2760] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 25 05:45:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:45:22.252858 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b1d32e-c598-4914-a5cd-e998ae005c23 None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:45:22.253152 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b1d32e-c598-4914-a5cd-e998ae005c23 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:45:22.253462 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b1d32e-c598-4914-a5cd-e998ae005c23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:45:22.272128 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-03b1d32e-c598-4914-a5cd-e998ae005c23 None None] MFA Rules not processed for user `1eae540242d94c10b0ad33960a855f08`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:45:22.273358 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-03b1d32e-c598-4914-a5cd-e998ae005c23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:45:22.281329 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1384/2761] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 25 05:45:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:45:22.285363 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da1b5b2d-e0d7-4cef-a8cf-3653a0ac73d0 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:45:22.295178 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da1b5b2d-e0d7-4cef-a8cf-3653a0ac73d0 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:45:22.299807 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a48700ea-3e66-487f-a320-dacdee650a8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:45:22.300469 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48700ea-3e66-487f-a320-dacdee650a8d admin admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:45:22.300752 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48700ea-3e66-487f-a320-dacdee650a8d admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:45:22.301080 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48700ea-3e66-487f-a320-dacdee650a8d admin admin] PATH_INFO: `/v3/domains` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:45:22.301875 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a48700ea-3e66-487f-a320-dacdee650a8d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:45:22.302631 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a48700ea-3e66-487f-a320-dacdee650a8d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:45:22.304815 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a48700ea-3e66-487f-a320-dacdee650a8d admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:45:22.320949 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1378/2762] 10.222.0.33 () {58 vars in 1152 bytes} [Mon Jul 25 05:45:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:45:22.322905 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b1d32e-c598-4914-a5cd-e998ae005c23 None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:45:22.332567 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b1d32e-c598-4914-a5cd-e998ae005c23 None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:45:22.337338 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b819a2b-0b91-4eeb-a88a-03b0caff455f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:45:22.338063 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b819a2b-0b91-4eeb-a88a-03b0caff455f admin admin] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:45:22.338349 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b819a2b-0b91-4eeb-a88a-03b0caff455f admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:45:22.338625 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b819a2b-0b91-4eeb-a88a-03b0caff455f admin admin] PATH_INFO: `/v3/projects` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:45:22.339315 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b819a2b-0b91-4eeb-a88a-03b0caff455f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:45:22.341227 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b819a2b-0b91-4eeb-a88a-03b0caff455f admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:45:22.368323 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1385/2763] 10.222.0.33 () {60 vars in 1152 bytes} [Mon Jul 25 05:45:22 2022] POST /identity/v3/projects => generated 369 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 05:45:22.373574 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a48700ea-3e66-487f-a320-dacdee650a8d admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:45:22.383635 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a48700ea-3e66-487f-a320-dacdee650a8d admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:45:22.439770 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56325029-c8fc-4fa0-9331-269cc213ea07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:45:22.440441 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56325029-c8fc-4fa0-9331-269cc213ea07 admin admin] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:45:22.440723 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56325029-c8fc-4fa0-9331-269cc213ea07 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:45:22.441029 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56325029-c8fc-4fa0-9331-269cc213ea07 admin admin] PATH_INFO: `/v3/users` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:45:22.441720 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56325029-c8fc-4fa0-9331-269cc213ea07 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:45:22.443624 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56325029-c8fc-4fa0-9331-269cc213ea07 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:45:22.466427 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1379/2764] 10.222.0.33 () {60 vars in 1143 bytes} [Mon Jul 25 05:45:22 2022] POST /identity/v3/users => generated 285 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 25 05:45:22.470425 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b819a2b-0b91-4eeb-a88a-03b0caff455f admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:45:22.482104 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b819a2b-0b91-4eeb-a88a-03b0caff455f admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:45:22.486727 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43bf1ae5-f4e4-4ce3-a004-09386f55ed0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:45:22.487383 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43bf1ae5-f4e4-4ce3-a004-09386f55ed0a admin admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:45:22.487667 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43bf1ae5-f4e4-4ce3-a004-09386f55ed0a admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:45:22.487950 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43bf1ae5-f4e4-4ce3-a004-09386f55ed0a admin admin] PATH_INFO: `/v3/roles` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:45:22.488677 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43bf1ae5-f4e4-4ce3-a004-09386f55ed0a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:45:22.490334 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43bf1ae5-f4e4-4ce3-a004-09386f55ed0a admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:45:22.503699 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1386/2765] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jul 25 05:45:22 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 05:45:22.507786 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56325029-c8fc-4fa0-9331-269cc213ea07 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:45:22.518416 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56325029-c8fc-4fa0-9331-269cc213ea07 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:45:22.523147 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea6d4bce-a534-4131-b8b0-12c815956f96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:45:22.523809 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6d4bce-a534-4131-b8b0-12c815956f96 admin admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:45:22.524129 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6d4bce-a534-4131-b8b0-12c815956f96 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:45:22.524415 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6d4bce-a534-4131-b8b0-12c815956f96 admin admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:45:22.525817 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea6d4bce-a534-4131-b8b0-12c815956f96 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:45:22.527413 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea6d4bce-a534-4131-b8b0-12c815956f96 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:45:22.537695 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1380/2766] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jul 25 05:45:22 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 05:45:22.541803 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43bf1ae5-f4e4-4ce3-a004-09386f55ed0a admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:45:22.555280 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43bf1ae5-f4e4-4ce3-a004-09386f55ed0a admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:45:22.560527 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d702cfd-e4af-460a-8e32-bc94bae22520 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:45:22.561434 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d702cfd-e4af-460a-8e32-bc94bae22520 admin admin] REQUEST_METHOD: `PUT` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:45:22.561717 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d702cfd-e4af-460a-8e32-bc94bae22520 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:45:22.561995 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d702cfd-e4af-460a-8e32-bc94bae22520 admin admin] PATH_INFO: `/v3/projects/354d7ebd6b454fb5a601a2a78a88561e/users/1574ee677bad49d190a0f82827d5aade/roles/5074e74e35f54d4b9bb415c34999a82a` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:45:22.565039 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:45:22.578785 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d702cfd-e4af-460a-8e32-bc94bae22520 admin admin] RBAC: Authorizing `identity:create_grant(project_id=354d7ebd6b454fb5a601a2a78a88561e, user_id=1574ee677bad49d190a0f82827d5aade, role_id=5074e74e35f54d4b9bb415c34999a82a)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:45:22.581469 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d702cfd-e4af-460a-8e32-bc94bae22520 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:45:22.594981 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1387/2767] 10.222.0.33 () {60 vars in 1482 bytes} [Mon Jul 25 05:45:22 2022] PUT /identity/v3/projects/354d7ebd6b454fb5a601a2a78a88561e/users/1574ee677bad49d190a0f82827d5aade/roles/5074e74e35f54d4b9bb415c34999a82a => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 05:45:22.600047 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe87a8b-489c-4f9c-8ab3-773f1062d77b None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:45:22.600342 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe87a8b-489c-4f9c-8ab3-773f1062d77b None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:45:22.600624 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe87a8b-489c-4f9c-8ab3-773f1062d77b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:45:22.618755 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-abe87a8b-489c-4f9c-8ab3-773f1062d77b None None] MFA Rules not processed for user `1eae540242d94c10b0ad33960a855f08`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:45:22.620242 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-abe87a8b-489c-4f9c-8ab3-773f1062d77b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:45:22.628520 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:45:22.680601 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1381/2768] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 25 05:45:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:45:22.690113 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d702cfd-e4af-460a-8e32-bc94bae22520 admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:45:22.692156 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-9d702cfd-e4af-460a-8e32-bc94bae22520 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:45:22.770518 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d702cfd-e4af-460a-8e32-bc94bae22520 admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:45:22.774255 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f68a8644-2de2-40d6-bab8-0605642ff443 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b2931200f99e43d79a50a9f6e02738e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92afba266cce4d1bad4134c351e74be3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:45:22.774918 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68a8644-2de2-40d6-bab8-0605642ff443 service neutron] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:45:22.776066 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68a8644-2de2-40d6-bab8-0605642ff443 service neutron] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:45:22.776066 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68a8644-2de2-40d6-bab8-0605642ff443 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:45:22.785278 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f68a8644-2de2-40d6-bab8-0605642ff443 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:45:22.786716 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f68a8644-2de2-40d6-bab8-0605642ff443 service neutron] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:45:22.796268 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1388/2769] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jul 25 05:45:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 05:45:26.980891 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaaedbad-004b-4bd3-9f7e-61e26dc41945 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:45:26.980891 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaaedbad-004b-4bd3-9f7e-61e26dc41945 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:45:26.980891 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaaedbad-004b-4bd3-9f7e-61e26dc41945 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:45:27.006804 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-aaaedbad-004b-4bd3-9f7e-61e26dc41945 None None] MFA Rules not processed for user `1574ee677bad49d190a0f82827d5aade`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:45:27.008093 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-aaaedbad-004b-4bd3-9f7e-61e26dc41945 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:45:27.074303 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1382/2770] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 25 05:45:26 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:45:27.078903 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f68a8644-2de2-40d6-bab8-0605642ff443 service neutron] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:45:27.089394 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f68a8644-2de2-40d6-bab8-0605642ff443 service neutron] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:45:27.094906 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a09426-2e21-4310-a654-0ec9ea9fa97d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:45:27.095682 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a09426-2e21-4310-a654-0ec9ea9fa97d admin admin] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:45:27.095968 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a09426-2e21-4310-a654-0ec9ea9fa97d admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:45:27.096246 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a09426-2e21-4310-a654-0ec9ea9fa97d admin admin] PATH_INFO: `/v3/projects` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:45:27.096985 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46a09426-2e21-4310-a654-0ec9ea9fa97d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:45:27.098924 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46a09426-2e21-4310-a654-0ec9ea9fa97d admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:45:27.129327 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1389/2771] 10.222.0.33 () {60 vars in 1152 bytes} [Mon Jul 25 05:45:27 2022] POST /identity/v3/projects => generated 371 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 05:45:27.130500 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaaedbad-004b-4bd3-9f7e-61e26dc41945 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:45:27.144271 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaaedbad-004b-4bd3-9f7e-61e26dc41945 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:45:27.203185 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d2e1cb-b5ae-488e-aef0-7d72ed7581c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:45:27.203872 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d2e1cb-b5ae-488e-aef0-7d72ed7581c9 admin admin] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:45:27.204159 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d2e1cb-b5ae-488e-aef0-7d72ed7581c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:45:27.204436 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d2e1cb-b5ae-488e-aef0-7d72ed7581c9 admin admin] PATH_INFO: `/v3/users` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:45:27.205175 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8d2e1cb-b5ae-488e-aef0-7d72ed7581c9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:45:27.207092 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8d2e1cb-b5ae-488e-aef0-7d72ed7581c9 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:45:27.230844 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1383/2772] 10.222.0.33 () {60 vars in 1143 bytes} [Mon Jul 25 05:45:27 2022] POST /identity/v3/users => generated 292 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 05:45:27.234880 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a09426-2e21-4310-a654-0ec9ea9fa97d admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:45:27.251593 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a09426-2e21-4310-a654-0ec9ea9fa97d admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:45:27.256366 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3808e381-3bed-4f3e-9776-e60b001fce56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:45:27.257130 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3808e381-3bed-4f3e-9776-e60b001fce56 admin admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:45:27.257444 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3808e381-3bed-4f3e-9776-e60b001fce56 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:45:27.257715 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3808e381-3bed-4f3e-9776-e60b001fce56 admin admin] PATH_INFO: `/v3/roles` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:45:27.258443 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3808e381-3bed-4f3e-9776-e60b001fce56 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:45:27.260692 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3808e381-3bed-4f3e-9776-e60b001fce56 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:45:27.271481 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1390/2773] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jul 25 05:45:27 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 05:45:27.276134 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d2e1cb-b5ae-488e-aef0-7d72ed7581c9 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:45:27.286963 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d2e1cb-b5ae-488e-aef0-7d72ed7581c9 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:45:27.292732 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c04e2554-6295-4fdb-ad10-d5fd3647d120 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:45:27.293656 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04e2554-6295-4fdb-ad10-d5fd3647d120 admin admin] REQUEST_METHOD: `PUT` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:45:27.293939 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04e2554-6295-4fdb-ad10-d5fd3647d120 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:45:27.294214 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04e2554-6295-4fdb-ad10-d5fd3647d120 admin admin] PATH_INFO: `/v3/projects/f647fa5b3b6f41b9a2bf80e202a2c629/users/62cb0487409e4f36b227f5315e914d40/roles/39e5c666285a42fe8fc2c3afca6260d1` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:45:27.311117 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c04e2554-6295-4fdb-ad10-d5fd3647d120 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f647fa5b3b6f41b9a2bf80e202a2c629, user_id=62cb0487409e4f36b227f5315e914d40, role_id=39e5c666285a42fe8fc2c3afca6260d1)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:45:27.313849 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c04e2554-6295-4fdb-ad10-d5fd3647d120 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:45:27.328155 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1384/2774] 10.222.0.33 () {60 vars in 1482 bytes} [Mon Jul 25 05:45:27 2022] PUT /identity/v3/projects/f647fa5b3b6f41b9a2bf80e202a2c629/users/62cb0487409e4f36b227f5315e914d40/roles/39e5c666285a42fe8fc2c3afca6260d1 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 05:45:27.332303 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3808e381-3bed-4f3e-9776-e60b001fce56 admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:45:27.343770 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3808e381-3bed-4f3e-9776-e60b001fce56 admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:45:27.440539 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-994c57d0-c966-4e53-84df-0f6faa9a34ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:45:27.442138 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994c57d0-c966-4e53-84df-0f6faa9a34ce admin admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:45:27.442138 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994c57d0-c966-4e53-84df-0f6faa9a34ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:45:27.442138 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994c57d0-c966-4e53-84df-0f6faa9a34ce admin admin] PATH_INFO: `/v3/roles` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:45:27.442832 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-994c57d0-c966-4e53-84df-0f6faa9a34ce admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:45:27.444550 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-994c57d0-c966-4e53-84df-0f6faa9a34ce admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:45:27.458851 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1391/2775] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jul 25 05:45:27 2022] GET /identity/v3/roles => generated 1420 bytes in 127 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 05:45:27.464567 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c04e2554-6295-4fdb-ad10-d5fd3647d120 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:45:27.488917 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c04e2554-6295-4fdb-ad10-d5fd3647d120 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:45:27.505267 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-802d10d6-b83b-4726-90b1-4b797a7dea97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:45:27.506167 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802d10d6-b83b-4726-90b1-4b797a7dea97 admin admin] REQUEST_METHOD: `PUT` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:45:27.506476 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802d10d6-b83b-4726-90b1-4b797a7dea97 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:45:27.506859 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802d10d6-b83b-4726-90b1-4b797a7dea97 admin admin] PATH_INFO: `/v3/projects/f647fa5b3b6f41b9a2bf80e202a2c629/users/62cb0487409e4f36b227f5315e914d40/roles/39e5c666285a42fe8fc2c3afca6260d1` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:45:27.509081 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-802d10d6-b83b-4726-90b1-4b797a7dea97 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f647fa5b3b6f41b9a2bf80e202a2c629, user_id=62cb0487409e4f36b227f5315e914d40, role_id=39e5c666285a42fe8fc2c3afca6260d1)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:45:27.512883 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-802d10d6-b83b-4726-90b1-4b797a7dea97 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:45:27.517999 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:45:27.533226 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1385/2776] 10.222.0.33 () {60 vars in 1482 bytes} [Mon Jul 25 05:45:27 2022] PUT /identity/v3/projects/f647fa5b3b6f41b9a2bf80e202a2c629/users/62cb0487409e4f36b227f5315e914d40/roles/39e5c666285a42fe8fc2c3afca6260d1 => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 05:45:30.326882 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-994c57d0-c966-4e53-84df-0f6faa9a34ce admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:45:30.329361 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-994c57d0-c966-4e53-84df-0f6faa9a34ce admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:45:30.406888 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:45:30.440310 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-994c57d0-c966-4e53-84df-0f6faa9a34ce admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:45:30.507476 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-123169e6-7040-4984-9fa8-335cc345c206 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b2931200f99e43d79a50a9f6e02738e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92afba266cce4d1bad4134c351e74be3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:45:30.509014 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123169e6-7040-4984-9fa8-335cc345c206 service neutron] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:45:30.509603 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123169e6-7040-4984-9fa8-335cc345c206 service neutron] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:45:30.510085 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123169e6-7040-4984-9fa8-335cc345c206 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:45:30.512735 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-123169e6-7040-4984-9fa8-335cc345c206 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:45:30.596164 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-123169e6-7040-4984-9fa8-335cc345c206 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:45:30.598494 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-123169e6-7040-4984-9fa8-335cc345c206 service neutron] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:45:30.611752 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1392/2777] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jul 25 05:45:30 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 285 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 05:45:32.648616 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0c6e12-3f18-436e-af67-c59383139750 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:45:32.649168 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0c6e12-3f18-436e-af67-c59383139750 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:45:32.649271 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0c6e12-3f18-436e-af67-c59383139750 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:45:32.709812 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-8c0c6e12-3f18-436e-af67-c59383139750 None None] MFA Rules not processed for user `62cb0487409e4f36b227f5315e914d40`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:45:32.711760 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-8c0c6e12-3f18-436e-af67-c59383139750 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:45:32.824464 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1386/2778] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 25 05:45:32 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:45:32.843539 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-123169e6-7040-4984-9fa8-335cc345c206 service neutron] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:45:32.845123 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-123169e6-7040-4984-9fa8-335cc345c206 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:45:33.030428 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-123169e6-7040-4984-9fa8-335cc345c206 service neutron] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:45:33.036982 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30312e02-1261-470c-bdfa-1f3a08ab9b47 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cd482a61e8a420db45bb532e5dae6e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92afba266cce4d1bad4134c351e74be3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:45:33.037977 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30312e02-1261-470c-bdfa-1f3a08ab9b47 service nova] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:45:33.038437 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30312e02-1261-470c-bdfa-1f3a08ab9b47 service nova] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:45:33.038881 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30312e02-1261-470c-bdfa-1f3a08ab9b47 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:45:33.056236 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:45:33.128996 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30312e02-1261-470c-bdfa-1f3a08ab9b47 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:45:33.136982 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30312e02-1261-470c-bdfa-1f3a08ab9b47 service nova] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:45:33.149839 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1393/2779] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jul 25 05:45:32 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 05:45:34.449801 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb2d34c-f0d0-49dd-980a-7b4216220091 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:45:34.450445 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb2d34c-f0d0-49dd-980a-7b4216220091 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:45:34.450445 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb2d34c-f0d0-49dd-980a-7b4216220091 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:45:34.469117 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-1cb2d34c-f0d0-49dd-980a-7b4216220091 None None] MFA Rules not processed for user `1574ee677bad49d190a0f82827d5aade`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:45:34.470268 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-1cb2d34c-f0d0-49dd-980a-7b4216220091 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:45:34.477665 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1387/2780] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 25 05:45:34 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:45:43.144137 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=21 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:45:44.462757 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=39 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:45:53.987513 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30312e02-1261-470c-bdfa-1f3a08ab9b47 service nova] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:45:53.989010 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-30312e02-1261-470c-bdfa-1f3a08ab9b47 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:45:54.057696 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30312e02-1261-470c-bdfa-1f3a08ab9b47 service nova] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:45:54.062945 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab80fea-5f27-450a-bf68-93f43e0ab0d7 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '317790016e7047fe96af73925de5d872', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92afba266cce4d1bad4134c351e74be3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:45:54.063596 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab80fea-5f27-450a-bf68-93f43e0ab0d7 service placement] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:45:54.063879 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab80fea-5f27-450a-bf68-93f43e0ab0d7 service placement] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:45:54.064152 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab80fea-5f27-450a-bf68-93f43e0ab0d7 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:45:54.065961 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-1ab80fea-5f27-450a-bf68-93f43e0ab0d7 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:45:54.077092 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ab80fea-5f27-450a-bf68-93f43e0ab0d7 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:45:54.079269 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ab80fea-5f27-450a-bf68-93f43e0ab0d7 service placement] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:45:54.088994 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1394/2781] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jul 25 05:45:53 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 05:46:03.512679 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba9de94-c9d8-417c-a5b9-ea3204a06ff5 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:46:03.513245 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba9de94-c9d8-417c-a5b9-ea3204a06ff5 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:46:03.513379 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba9de94-c9d8-417c-a5b9-ea3204a06ff5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:46:03.531014 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-1ba9de94-c9d8-417c-a5b9-ea3204a06ff5 None None] MFA Rules not processed for user `6973623dc52c402ab51dbf1a7169d3e9`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:46:03.532831 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-1ba9de94-c9d8-417c-a5b9-ea3204a06ff5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:46:03.586883 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1388/2782] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 25 05:46:03 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:46:04.084693 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=32 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:46:11.590891 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab80fea-5f27-450a-bf68-93f43e0ab0d7 service placement] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:46:11.600936 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab80fea-5f27-450a-bf68-93f43e0ab0d7 service placement] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:46:11.604896 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40256274-c4bb-424e-b24d-95eee208d41d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b2931200f99e43d79a50a9f6e02738e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92afba266cce4d1bad4134c351e74be3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:46:11.605555 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40256274-c4bb-424e-b24d-95eee208d41d service neutron] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:46:11.605845 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40256274-c4bb-424e-b24d-95eee208d41d service neutron] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:46:11.606127 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40256274-c4bb-424e-b24d-95eee208d41d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:46:11.616457 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40256274-c4bb-424e-b24d-95eee208d41d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:46:11.617948 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40256274-c4bb-424e-b24d-95eee208d41d service neutron] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:46:11.629387 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1395/2783] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jul 25 05:46:11 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 25 05:46:13.580948 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=25 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:46:21.624351 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=6 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:47:08.151259 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba9de94-c9d8-417c-a5b9-ea3204a06ff5 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:47:08.161195 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba9de94-c9d8-417c-a5b9-ea3204a06ff5 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:47:08.164865 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a56f4dcb-470c-48fb-9834-2170ab0d3152 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b2931200f99e43d79a50a9f6e02738e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92afba266cce4d1bad4134c351e74be3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:47:08.165506 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56f4dcb-470c-48fb-9834-2170ab0d3152 service neutron] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:47:08.165821 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56f4dcb-470c-48fb-9834-2170ab0d3152 service neutron] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:47:08.166721 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56f4dcb-470c-48fb-9834-2170ab0d3152 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:47:08.168673 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-a56f4dcb-470c-48fb-9834-2170ab0d3152 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:47:08.183582 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a56f4dcb-470c-48fb-9834-2170ab0d3152 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:47:08.185053 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a56f4dcb-470c-48fb-9834-2170ab0d3152 service neutron] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:47:08.197266 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1389/2784] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jul 25 05:47:08 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 05:47:08.636606 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977f5d58-6bf6-454e-a0c0-a60282aab93e None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:47:08.637167 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977f5d58-6bf6-454e-a0c0-a60282aab93e None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:47:08.637283 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977f5d58-6bf6-454e-a0c0-a60282aab93e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:47:08.654781 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-977f5d58-6bf6-454e-a0c0-a60282aab93e None None] MFA Rules not processed for user `3352531be1b54f76b864383ccc2e5ec2`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:47:08.655991 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-977f5d58-6bf6-454e-a0c0-a60282aab93e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:47:08.712438 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1396/2785] 10.222.0.33 () {58 vars in 1009 bytes} [Mon Jul 25 05:47:08 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:47:08.765343 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a56f4dcb-470c-48fb-9834-2170ab0d3152 service neutron] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:47:08.766840 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-a56f4dcb-470c-48fb-9834-2170ab0d3152 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:47:08.841515 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a56f4dcb-470c-48fb-9834-2170ab0d3152 service neutron] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:47:08.847259 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d51eef6-ede4-410b-b704-6785d1b8f61c service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb135a8d5d7444cfbfa336115ba2ba44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92afba266cce4d1bad4134c351e74be3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:47:08.847998 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d51eef6-ede4-410b-b704-6785d1b8f61c service swift] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:47:08.848263 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d51eef6-ede4-410b-b704-6785d1b8f61c service swift] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:47:08.848542 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d51eef6-ede4-410b-b704-6785d1b8f61c service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:47:08.858448 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d51eef6-ede4-410b-b704-6785d1b8f61c service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:47:08.860025 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d51eef6-ede4-410b-b704-6785d1b8f61c service swift] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:47:08.868286 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1390/2786] 10.222.0.33 () {60 vars in 1384 bytes} [Mon Jul 25 05:47:08 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jul 25 05:47:10.616818 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-977f5d58-6bf6-454e-a0c0-a60282aab93e None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:47:10.630495 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-977f5d58-6bf6-454e-a0c0-a60282aab93e None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:47:10.635056 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c5b9d94-8d44-4fd2-bd90-3176201eda2b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b2931200f99e43d79a50a9f6e02738e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92afba266cce4d1bad4134c351e74be3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:47:10.635714 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5b9d94-8d44-4fd2-bd90-3176201eda2b service neutron] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:47:10.635995 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5b9d94-8d44-4fd2-bd90-3176201eda2b service neutron] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:47:10.636269 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5b9d94-8d44-4fd2-bd90-3176201eda2b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:47:10.638291 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-2c5b9d94-8d44-4fd2-bd90-3176201eda2b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:47:10.653294 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c5b9d94-8d44-4fd2-bd90-3176201eda2b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:47:10.654770 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c5b9d94-8d44-4fd2-bd90-3176201eda2b service neutron] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:47:10.670654 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1397/2787] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jul 25 05:47:10 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 05:47:18.864908 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=38 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:47:20.661466 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=31 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:47:25.942074 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7004ad7-ed57-44bd-ac9c-029ecb9eb2f0 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:47:25.942610 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7004ad7-ed57-44bd-ac9c-029ecb9eb2f0 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:47:25.942712 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7004ad7-ed57-44bd-ac9c-029ecb9eb2f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:47:25.996857 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-f7004ad7-ed57-44bd-ac9c-029ecb9eb2f0 None None] MFA Rules not processed for user `3352531be1b54f76b864383ccc2e5ec2`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:47:25.998178 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-f7004ad7-ed57-44bd-ac9c-029ecb9eb2f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:47:26.015004 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1391/2788] 10.222.0.33 () {58 vars in 1009 bytes} [Mon Jul 25 05:47:25 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 74 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:47:26.027633 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c5b9d94-8d44-4fd2-bd90-3176201eda2b service neutron] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:47:26.048173 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c5b9d94-8d44-4fd2-bd90-3176201eda2b service neutron] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:47:26.053139 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ca4985-4ae6-424a-b8d2-7d388921dd1e service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb135a8d5d7444cfbfa336115ba2ba44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92afba266cce4d1bad4134c351e74be3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:47:26.053778 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ca4985-4ae6-424a-b8d2-7d388921dd1e service swift] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:47:26.054062 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ca4985-4ae6-424a-b8d2-7d388921dd1e service swift] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:47:26.054336 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ca4985-4ae6-424a-b8d2-7d388921dd1e service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:47:26.063054 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22ca4985-4ae6-424a-b8d2-7d388921dd1e service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:47:26.064620 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22ca4985-4ae6-424a-b8d2-7d388921dd1e service swift] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:47:26.073163 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1398/2789] 10.222.0.33 () {60 vars in 1384 bytes} [Mon Jul 25 05:47:26 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 25 05:47:36.012163 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=18 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:47:36.069256 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=10 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:47:38.378114 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7004ad7-ed57-44bd-ac9c-029ecb9eb2f0 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:47:38.388135 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7004ad7-ed57-44bd-ac9c-029ecb9eb2f0 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:47:38.391753 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-126e86b4-9916-41cd-9014-5230f459f8af service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b2931200f99e43d79a50a9f6e02738e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92afba266cce4d1bad4134c351e74be3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:47:38.392405 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126e86b4-9916-41cd-9014-5230f459f8af service neutron] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:47:38.392692 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126e86b4-9916-41cd-9014-5230f459f8af service neutron] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:47:38.393026 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126e86b4-9916-41cd-9014-5230f459f8af service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:47:38.394839 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-126e86b4-9916-41cd-9014-5230f459f8af service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:47:38.457572 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-126e86b4-9916-41cd-9014-5230f459f8af service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:47:38.459014 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-126e86b4-9916-41cd-9014-5230f459f8af service neutron] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:47:38.470383 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1392/2790] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jul 25 05:47:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 05:47:40.779322 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ca4985-4ae6-424a-b8d2-7d388921dd1e service swift] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:47:40.789274 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ca4985-4ae6-424a-b8d2-7d388921dd1e service swift] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:47:40.793587 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9686f67-6f66-4cd2-bd62-754079b431df service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b2931200f99e43d79a50a9f6e02738e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92afba266cce4d1bad4134c351e74be3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:47:40.794227 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9686f67-6f66-4cd2-bd62-754079b431df service neutron] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:47:40.794513 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9686f67-6f66-4cd2-bd62-754079b431df service neutron] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:47:40.794793 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9686f67-6f66-4cd2-bd62-754079b431df service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:47:40.796533 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-a9686f67-6f66-4cd2-bd62-754079b431df service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:47:40.809675 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9686f67-6f66-4cd2-bd62-754079b431df service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:47:40.811151 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9686f67-6f66-4cd2-bd62-754079b431df service neutron] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:47:40.820271 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1399/2791] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jul 25 05:47:40 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 05:47:42.104602 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f43b553-d3f8-498e-8ebc-f7e2e35b69ce None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:47:42.105154 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f43b553-d3f8-498e-8ebc-f7e2e35b69ce None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:47:42.105347 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f43b553-d3f8-498e-8ebc-f7e2e35b69ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:47:42.123798 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-7f43b553-d3f8-498e-8ebc-f7e2e35b69ce None None] MFA Rules not processed for user `3352531be1b54f76b864383ccc2e5ec2`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:47:42.125063 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-7f43b553-d3f8-498e-8ebc-f7e2e35b69ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:47:42.132301 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1393/2792] 10.222.0.33 () {58 vars in 967 bytes} [Mon Jul 25 05:47:42 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:47:42.145460 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9686f67-6f66-4cd2-bd62-754079b431df service neutron] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:47:42.158397 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9686f67-6f66-4cd2-bd62-754079b431df service neutron] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:47:42.163607 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96cb93b5-4fa1-4a0c-98b1-93056628aaa5 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb135a8d5d7444cfbfa336115ba2ba44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92afba266cce4d1bad4134c351e74be3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:47:42.164436 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96cb93b5-4fa1-4a0c-98b1-93056628aaa5 service swift] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:47:42.164736 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96cb93b5-4fa1-4a0c-98b1-93056628aaa5 service swift] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:47:42.165129 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96cb93b5-4fa1-4a0c-98b1-93056628aaa5 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:47:42.174143 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96cb93b5-4fa1-4a0c-98b1-93056628aaa5 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:47:42.175764 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96cb93b5-4fa1-4a0c-98b1-93056628aaa5 service swift] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:47:42.184465 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1400/2793] 10.222.0.33 () {60 vars in 1384 bytes} [Mon Jul 25 05:47:42 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 25 05:47:45.058269 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f43b553-d3f8-498e-8ebc-f7e2e35b69ce None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:47:45.059787 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-7f43b553-d3f8-498e-8ebc-f7e2e35b69ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:47:45.075089 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f43b553-d3f8-498e-8ebc-f7e2e35b69ce None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:47:45.079830 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3bd9c76-aaa8-4235-bee9-b5e60e5dbec6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:47:45.080704 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3bd9c76-aaa8-4235-bee9-b5e60e5dbec6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:47:45.081049 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3bd9c76-aaa8-4235-bee9-b5e60e5dbec6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:47:45.081335 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3bd9c76-aaa8-4235-bee9-b5e60e5dbec6 admin admin] PATH_INFO: `/v3/users/1f815f111bf74739969736acea131927` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:47:45.082551 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3bd9c76-aaa8-4235-bee9-b5e60e5dbec6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1f815f111bf74739969736acea131927)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:47:45.084523 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3bd9c76-aaa8-4235-bee9-b5e60e5dbec6 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:47:45.132065 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-e3bd9c76-aaa8-4235-bee9-b5e60e5dbec6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1f815f111bf74739969736acea131927'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:47:45.132757 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-e3bd9c76-aaa8-4235-bee9-b5e60e5dbec6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1f815f111bf74739969736acea131927'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:47:45.140479 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-e3bd9c76-aaa8-4235-bee9-b5e60e5dbec6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1f815f111bf74739969736acea131927'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:47:45.153392 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-e3bd9c76-aaa8-4235-bee9-b5e60e5dbec6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1f815f111bf74739969736acea131927'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:47:45.154136 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-e3bd9c76-aaa8-4235-bee9-b5e60e5dbec6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1f815f111bf74739969736acea131927'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:47:45.169327 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1394/2794] 10.222.0.33 () {58 vars in 1223 bytes} [Mon Jul 25 05:47:45 2022] DELETE /identity/v3/users/1f815f111bf74739969736acea131927 => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 05:47:45.340638 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96cb93b5-4fa1-4a0c-98b1-93056628aaa5 service swift] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:47:45.353230 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96cb93b5-4fa1-4a0c-98b1-93056628aaa5 service swift] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:47:45.409827 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-856959d2-7051-4d98-8e27-f27e20beae9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:47:45.410685 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856959d2-7051-4d98-8e27-f27e20beae9f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:47:45.410972 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856959d2-7051-4d98-8e27-f27e20beae9f admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:47:45.411247 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856959d2-7051-4d98-8e27-f27e20beae9f admin admin] PATH_INFO: `/v3/projects/ee638472255a48d2bdffa17a4ff22339` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:47:45.412419 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-856959d2-7051-4d98-8e27-f27e20beae9f admin admin] RBAC: Authorizing `identity:delete_project(project_id=ee638472255a48d2bdffa17a4ff22339)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:47:45.414336 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-856959d2-7051-4d98-8e27-f27e20beae9f admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:47:45.460035 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-856959d2-7051-4d98-8e27-f27e20beae9f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ee638472255a48d2bdffa17a4ff22339', 'request_id': 'req-856959d2-7051-4d98-8e27-f27e20beae9f'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:47:45.466015 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-856959d2-7051-4d98-8e27-f27e20beae9f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ee638472255a48d2bdffa17a4ff22339', 'request_id': 'req-856959d2-7051-4d98-8e27-f27e20beae9f'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:47:45.471956 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-856959d2-7051-4d98-8e27-f27e20beae9f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ee638472255a48d2bdffa17a4ff22339', 'request_id': 'req-856959d2-7051-4d98-8e27-f27e20beae9f'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:47:45.484314 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-856959d2-7051-4d98-8e27-f27e20beae9f admin admin] The token cache is being invalidate because project ee638472255a48d2bdffa17a4ff22339 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80248) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 05:47:45.484599 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-856959d2-7051-4d98-8e27-f27e20beae9f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:47:45.485496 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-856959d2-7051-4d98-8e27-f27e20beae9f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:47:45.487034 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1401/2795] 10.222.0.33 () {58 vars in 1232 bytes} [Mon Jul 25 05:47:45 2022] DELETE /identity/v3/projects/ee638472255a48d2bdffa17a4ff22339 => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 05:47:45.491026 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3bd9c76-aaa8-4235-bee9-b5e60e5dbec6 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:47:45.492446 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-e3bd9c76-aaa8-4235-bee9-b5e60e5dbec6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:47:45.505715 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3bd9c76-aaa8-4235-bee9-b5e60e5dbec6 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:47:45.561933 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aab4499-a3b8-438f-82d3-609abcfe4344 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:47:45.562802 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aab4499-a3b8-438f-82d3-609abcfe4344 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:47:45.563086 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aab4499-a3b8-438f-82d3-609abcfe4344 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:47:45.563364 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aab4499-a3b8-438f-82d3-609abcfe4344 admin admin] PATH_INFO: `/v3/users/4b81270577ff4bb3a8b8b8479a59efd4` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:47:45.564561 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aab4499-a3b8-438f-82d3-609abcfe4344 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4b81270577ff4bb3a8b8b8479a59efd4)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:47:45.566505 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aab4499-a3b8-438f-82d3-609abcfe4344 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:47:45.567981 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:47:45.616994 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-7aab4499-a3b8-438f-82d3-609abcfe4344 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4b81270577ff4bb3a8b8b8479a59efd4'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:47:45.617584 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-7aab4499-a3b8-438f-82d3-609abcfe4344 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4b81270577ff4bb3a8b8b8479a59efd4'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:47:45.624996 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-7aab4499-a3b8-438f-82d3-609abcfe4344 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4b81270577ff4bb3a8b8b8479a59efd4'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:47:45.637003 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-7aab4499-a3b8-438f-82d3-609abcfe4344 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4b81270577ff4bb3a8b8b8479a59efd4'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:47:45.637672 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-7aab4499-a3b8-438f-82d3-609abcfe4344 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4b81270577ff4bb3a8b8b8479a59efd4'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:47:45.652205 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1395/2796] 10.222.0.33 () {58 vars in 1223 bytes} [Mon Jul 25 05:47:45 2022] DELETE /identity/v3/users/4b81270577ff4bb3a8b8b8479a59efd4 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 05:47:45.826314 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-856959d2-7051-4d98-8e27-f27e20beae9f admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:47:45.827860 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-856959d2-7051-4d98-8e27-f27e20beae9f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:47:45.842660 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-856959d2-7051-4d98-8e27-f27e20beae9f admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:47:45.901734 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d82d46d-e0d3-417e-b07c-ee4a63785c96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:47:45.902525 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d82d46d-e0d3-417e-b07c-ee4a63785c96 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:47:45.902814 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d82d46d-e0d3-417e-b07c-ee4a63785c96 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:47:45.903096 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d82d46d-e0d3-417e-b07c-ee4a63785c96 admin admin] PATH_INFO: `/v3/projects/e7503d2734414a67962cccd71e3b97e5` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:47:45.904279 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d82d46d-e0d3-417e-b07c-ee4a63785c96 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e7503d2734414a67962cccd71e3b97e5)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:47:45.906249 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d82d46d-e0d3-417e-b07c-ee4a63785c96 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:47:45.912368 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:47:45.953775 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-1d82d46d-e0d3-417e-b07c-ee4a63785c96 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e7503d2734414a67962cccd71e3b97e5', 'request_id': 'req-1d82d46d-e0d3-417e-b07c-ee4a63785c96'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:47:45.959296 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-1d82d46d-e0d3-417e-b07c-ee4a63785c96 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e7503d2734414a67962cccd71e3b97e5', 'request_id': 'req-1d82d46d-e0d3-417e-b07c-ee4a63785c96'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:47:45.966054 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-1d82d46d-e0d3-417e-b07c-ee4a63785c96 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e7503d2734414a67962cccd71e3b97e5', 'request_id': 'req-1d82d46d-e0d3-417e-b07c-ee4a63785c96'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:47:45.978366 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-1d82d46d-e0d3-417e-b07c-ee4a63785c96 admin admin] The token cache is being invalidate because project e7503d2734414a67962cccd71e3b97e5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80248) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 05:47:45.978677 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-1d82d46d-e0d3-417e-b07c-ee4a63785c96 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:47:45.979301 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-1d82d46d-e0d3-417e-b07c-ee4a63785c96 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:47:45.980845 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1402/2797] 10.222.0.33 () {58 vars in 1232 bytes} [Mon Jul 25 05:47:45 2022] DELETE /identity/v3/projects/e7503d2734414a67962cccd71e3b97e5 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 05:47:55.647472 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=26,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:47:55.974972 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=22,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:48:16.793897 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aab4499-a3b8-438f-82d3-609abcfe4344 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:48:16.795282 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-7aab4499-a3b8-438f-82d3-609abcfe4344 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:48:16.876895 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aab4499-a3b8-438f-82d3-609abcfe4344 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:48:16.926865 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-889a85f1-b90e-4725-9539-4c4b977a6aed service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db57b478c25643dbaf936f90f6e17d0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92afba266cce4d1bad4134c351e74be3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:48:16.927525 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889a85f1-b90e-4725-9539-4c4b977a6aed service cinder] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:48:16.927811 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889a85f1-b90e-4725-9539-4c4b977a6aed service cinder] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:48:16.928087 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889a85f1-b90e-4725-9539-4c4b977a6aed service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:48:16.930168 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-889a85f1-b90e-4725-9539-4c4b977a6aed service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:48:16.989597 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-889a85f1-b90e-4725-9539-4c4b977a6aed service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:48:16.991162 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-889a85f1-b90e-4725-9539-4c4b977a6aed service cinder] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:48:17.001897 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1396/2798] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jul 25 05:48:16 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 208 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 05:48:19.759824 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d82d46d-e0d3-417e-b07c-ee4a63785c96 admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:48:19.761340 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-1d82d46d-e0d3-417e-b07c-ee4a63785c96 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:48:19.846244 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d82d46d-e0d3-417e-b07c-ee4a63785c96 admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:48:19.853386 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d803b636-a8ea-4536-9b81-52950171ba71 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cd482a61e8a420db45bb532e5dae6e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92afba266cce4d1bad4134c351e74be3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:48:19.854069 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d803b636-a8ea-4536-9b81-52950171ba71 service nova] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:48:19.854362 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d803b636-a8ea-4536-9b81-52950171ba71 service nova] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:48:19.854646 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d803b636-a8ea-4536-9b81-52950171ba71 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:48:19.856583 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-d803b636-a8ea-4536-9b81-52950171ba71 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:48:19.873691 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d803b636-a8ea-4536-9b81-52950171ba71 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:48:19.875355 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d803b636-a8ea-4536-9b81-52950171ba71 service nova] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:48:19.888431 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1403/2799] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jul 25 05:48:19 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 05:48:24.934782 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-889a85f1-b90e-4725-9539-4c4b977a6aed service cinder] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:48:24.936256 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-889a85f1-b90e-4725-9539-4c4b977a6aed service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:48:24.999769 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-889a85f1-b90e-4725-9539-4c4b977a6aed service cinder] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:48:25.003644 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29347f91-5545-4450-86bf-572e124b34a1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b2931200f99e43d79a50a9f6e02738e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92afba266cce4d1bad4134c351e74be3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:48:25.004304 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29347f91-5545-4450-86bf-572e124b34a1 service neutron] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:48:25.004593 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29347f91-5545-4450-86bf-572e124b34a1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:48:25.004913 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29347f91-5545-4450-86bf-572e124b34a1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:48:25.006723 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-29347f91-5545-4450-86bf-572e124b34a1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:48:25.018885 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:48:25.083365 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29347f91-5545-4450-86bf-572e124b34a1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:48:25.084531 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29347f91-5545-4450-86bf-572e124b34a1 service neutron] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:48:25.095693 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1397/2800] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jul 25 05:48:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 161 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 25 05:48:27.883203 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d636f6c-7ecf-40fc-8cdd-a289115ce303 None None] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:48:27.884100 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d636f6c-7ecf-40fc-8cdd-a289115ce303 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:48:27.884391 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d636f6c-7ecf-40fc-8cdd-a289115ce303 None None] PATH_INFO: `/` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:48:27.885996 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1404/2801] 10.222.0.33 () {54 vars in 921 bytes} [Mon Jul 25 05:48:27 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:48:27.893019 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b8a8c7-f07e-413b-a823-1552459a5025 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:48:27.893947 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b8a8c7-f07e-413b-a823-1552459a5025 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:48:27.894395 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b8a8c7-f07e-413b-a823-1552459a5025 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:48:27.948629 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-c1b8a8c7-f07e-413b-a823-1552459a5025 None None] MFA Rules not processed for user `b2931200f99e43d79a50a9f6e02738e6`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:48:27.949925 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-c1b8a8c7-f07e-413b-a823-1552459a5025 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:48:27.957934 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1398/2802] 10.222.0.33 () {58 vars in 1018 bytes} [Mon Jul 25 05:48:27 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 66 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:48:27.983541 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d636f6c-7ecf-40fc-8cdd-a289115ce303 None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:48:27.995022 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d636f6c-7ecf-40fc-8cdd-a289115ce303 None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:48:27.999571 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-723b422f-5726-48d5-ae17-1e49e9c6c7db service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b2931200f99e43d79a50a9f6e02738e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92afba266cce4d1bad4134c351e74be3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:48:28.000241 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723b422f-5726-48d5-ae17-1e49e9c6c7db service neutron] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:48:28.000993 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723b422f-5726-48d5-ae17-1e49e9c6c7db service neutron] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:48:28.000993 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723b422f-5726-48d5-ae17-1e49e9c6c7db service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:48:28.010830 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-723b422f-5726-48d5-ae17-1e49e9c6c7db service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:48:28.012257 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-723b422f-5726-48d5-ae17-1e49e9c6c7db service neutron] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:48:28.021064 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1405/2803] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jul 25 05:48:27 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 05:48:28.319980 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b8a8c7-f07e-413b-a823-1552459a5025 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:48:28.321078 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-c1b8a8c7-f07e-413b-a823-1552459a5025 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:48:28.402353 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-723b422f-5726-48d5-ae17-1e49e9c6c7db service neutron] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:48:28.404343 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-723b422f-5726-48d5-ae17-1e49e9c6c7db service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:48:28.440555 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b8a8c7-f07e-413b-a823-1552459a5025 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:48:28.443283 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-723b422f-5726-48d5-ae17-1e49e9c6c7db service neutron] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:48:28.446613 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae1041f-7a10-459c-9e0a-e155f4b84733 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '317790016e7047fe96af73925de5d872', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92afba266cce4d1bad4134c351e74be3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:48:28.447389 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae1041f-7a10-459c-9e0a-e155f4b84733 service placement] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:48:28.447674 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae1041f-7a10-459c-9e0a-e155f4b84733 service placement] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:48:28.447975 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae1041f-7a10-459c-9e0a-e155f4b84733 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:48:28.450115 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-7ae1041f-7a10-459c-9e0a-e155f4b84733 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:48:28.450875 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e31df6-2956-41ba-8a04-4c24aa45398d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:48:28.452280 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e31df6-2956-41ba-8a04-4c24aa45398d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:48:28.452959 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e31df6-2956-41ba-8a04-4c24aa45398d admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:48:28.454053 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e31df6-2956-41ba-8a04-4c24aa45398d admin admin] PATH_INFO: `/v3/users/6973623dc52c402ab51dbf1a7169d3e9` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:48:28.456260 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78e31df6-2956-41ba-8a04-4c24aa45398d admin admin] RBAC: Authorizing `identity:delete_user(user_id=6973623dc52c402ab51dbf1a7169d3e9)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:48:28.459299 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78e31df6-2956-41ba-8a04-4c24aa45398d admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:48:28.463437 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ae1041f-7a10-459c-9e0a-e155f4b84733 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:48:28.465150 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ae1041f-7a10-459c-9e0a-e155f4b84733 service placement] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:48:28.476175 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1399/2804] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jul 25 05:48:28 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 157 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 05:48:28.477039 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae1041f-7a10-459c-9e0a-e155f4b84733 service placement] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:48:28.490182 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae1041f-7a10-459c-9e0a-e155f4b84733 service placement] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:48:28.497906 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14689ff8-e816-4d53-8e00-d453425f41d0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cd482a61e8a420db45bb532e5dae6e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92afba266cce4d1bad4134c351e74be3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:48:28.498715 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14689ff8-e816-4d53-8e00-d453425f41d0 service nova] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:48:28.499114 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14689ff8-e816-4d53-8e00-d453425f41d0 service nova] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:48:28.499463 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14689ff8-e816-4d53-8e00-d453425f41d0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:48:28.501565 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-14689ff8-e816-4d53-8e00-d453425f41d0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:48:28.517769 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14689ff8-e816-4d53-8e00-d453425f41d0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:48:28.519601 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14689ff8-e816-4d53-8e00-d453425f41d0 service nova] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:48:28.523695 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-78e31df6-2956-41ba-8a04-4c24aa45398d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6973623dc52c402ab51dbf1a7169d3e9'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:48:28.531641 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1400/2805] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jul 25 05:48:28 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 05:48:28.536689 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-78e31df6-2956-41ba-8a04-4c24aa45398d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6973623dc52c402ab51dbf1a7169d3e9'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:48:28.538093 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-78e31df6-2956-41ba-8a04-4c24aa45398d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6973623dc52c402ab51dbf1a7169d3e9'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:48:28.549379 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-78e31df6-2956-41ba-8a04-4c24aa45398d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6973623dc52c402ab51dbf1a7169d3e9'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:48:28.550130 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-78e31df6-2956-41ba-8a04-4c24aa45398d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6973623dc52c402ab51dbf1a7169d3e9'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:48:28.563775 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1406/2806] 10.222.0.33 () {58 vars in 1223 bytes} [Mon Jul 25 05:48:28 2022] DELETE /identity/v3/users/6973623dc52c402ab51dbf1a7169d3e9 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 25 05:48:28.735491 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14689ff8-e816-4d53-8e00-d453425f41d0 service nova] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:48:28.747438 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14689ff8-e816-4d53-8e00-d453425f41d0 service nova] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:48:28.810357 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:48:28.817525 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63914167-3201-41b6-ba88-d93dc29d3716 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:48:28.818544 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63914167-3201-41b6-ba88-d93dc29d3716 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:48:28.819030 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63914167-3201-41b6-ba88-d93dc29d3716 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:48:28.819378 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63914167-3201-41b6-ba88-d93dc29d3716 admin admin] PATH_INFO: `/v3/projects/07879d074bee4b36b1f2f661dc4aeb7f` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:48:28.820664 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63914167-3201-41b6-ba88-d93dc29d3716 admin admin] RBAC: Authorizing `identity:delete_project(project_id=07879d074bee4b36b1f2f661dc4aeb7f)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:48:28.822697 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63914167-3201-41b6-ba88-d93dc29d3716 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:48:28.870931 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-63914167-3201-41b6-ba88-d93dc29d3716 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '07879d074bee4b36b1f2f661dc4aeb7f', 'request_id': 'req-63914167-3201-41b6-ba88-d93dc29d3716'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:48:28.877752 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-63914167-3201-41b6-ba88-d93dc29d3716 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '07879d074bee4b36b1f2f661dc4aeb7f', 'request_id': 'req-63914167-3201-41b6-ba88-d93dc29d3716'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:48:28.884266 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-63914167-3201-41b6-ba88-d93dc29d3716 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '07879d074bee4b36b1f2f661dc4aeb7f', 'request_id': 'req-63914167-3201-41b6-ba88-d93dc29d3716'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:48:28.897288 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-63914167-3201-41b6-ba88-d93dc29d3716 admin admin] The token cache is being invalidate because project 07879d074bee4b36b1f2f661dc4aeb7f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80249) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 05:48:28.898337 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-63914167-3201-41b6-ba88-d93dc29d3716 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:48:28.899168 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-63914167-3201-41b6-ba88-d93dc29d3716 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:48:28.900721 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1401/2807] 10.222.0.33 () {58 vars in 1232 bytes} [Mon Jul 25 05:48:28 2022] DELETE /identity/v3/projects/07879d074bee4b36b1f2f661dc4aeb7f => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 05:48:28.905637 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b74c8e-a6cd-4856-a98f-e5be4c1ecd87 None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:48:28.906493 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b74c8e-a6cd-4856-a98f-e5be4c1ecd87 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:48:28.906775 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b74c8e-a6cd-4856-a98f-e5be4c1ecd87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:48:28.925032 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-19b74c8e-a6cd-4856-a98f-e5be4c1ecd87 None None] MFA Rules not processed for user `1eae540242d94c10b0ad33960a855f08`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:48:28.926250 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-19b74c8e-a6cd-4856-a98f-e5be4c1ecd87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:48:28.967154 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:48:28.988632 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1407/2808] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 25 05:48:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:48:28.996903 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982a2f28-4ca2-4b51-b5b0-d47aa7bad939 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:48:28.997200 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982a2f28-4ca2-4b51-b5b0-d47aa7bad939 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:48:28.997481 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982a2f28-4ca2-4b51-b5b0-d47aa7bad939 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:48:29.016431 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-982a2f28-4ca2-4b51-b5b0-d47aa7bad939 None None] MFA Rules not processed for user `1eae540242d94c10b0ad33960a855f08`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:48:29.017611 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-982a2f28-4ca2-4b51-b5b0-d47aa7bad939 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:48:29.025825 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1402/2809] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 25 05:48:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:48:29.030019 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b74c8e-a6cd-4856-a98f-e5be4c1ecd87 None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:48:29.040093 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b74c8e-a6cd-4856-a98f-e5be4c1ecd87 None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:48:29.044633 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b8db446-79a5-41ff-b4cf-d1182c6e03e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:48:29.045315 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8db446-79a5-41ff-b4cf-d1182c6e03e8 admin admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:48:29.045605 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8db446-79a5-41ff-b4cf-d1182c6e03e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:48:29.045884 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8db446-79a5-41ff-b4cf-d1182c6e03e8 admin admin] PATH_INFO: `/v3/domains` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:48:29.046616 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b8db446-79a5-41ff-b4cf-d1182c6e03e8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:48:29.046925 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b8db446-79a5-41ff-b4cf-d1182c6e03e8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:48:29.048543 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b8db446-79a5-41ff-b4cf-d1182c6e03e8 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:48:29.061142 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1408/2810] 10.222.0.33 () {58 vars in 1152 bytes} [Mon Jul 25 05:48:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:48:29.065498 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-982a2f28-4ca2-4b51-b5b0-d47aa7bad939 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:48:29.075208 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-982a2f28-4ca2-4b51-b5b0-d47aa7bad939 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:48:29.079790 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b22fec-9941-4f36-89df-acbcef14e96a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:48:29.080440 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b22fec-9941-4f36-89df-acbcef14e96a admin admin] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:48:29.080742 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b22fec-9941-4f36-89df-acbcef14e96a admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:48:29.081168 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b22fec-9941-4f36-89df-acbcef14e96a admin admin] PATH_INFO: `/v3/projects` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:48:29.081865 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29b22fec-9941-4f36-89df-acbcef14e96a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:48:29.083758 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29b22fec-9941-4f36-89df-acbcef14e96a admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:48:29.126646 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1403/2811] 10.222.0.33 () {60 vars in 1152 bytes} [Mon Jul 25 05:48:29 2022] POST /identity/v3/projects => generated 371 bytes in 61 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 05:48:29.130841 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b8db446-79a5-41ff-b4cf-d1182c6e03e8 admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:48:29.138037 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b049c1c6-0c0d-41d1-a53b-ade24436e2e3 None None] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:48:29.138360 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b049c1c6-0c0d-41d1-a53b-ade24436e2e3 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:48:29.138644 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b049c1c6-0c0d-41d1-a53b-ade24436e2e3 None None] PATH_INFO: `/` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:48:29.140022 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1404/2812] 10.222.0.33 () {54 vars in 921 bytes} [Mon Jul 25 05:48:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:48:29.145588 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b8db446-79a5-41ff-b4cf-d1182c6e03e8 admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:48:29.148196 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cacb2fa7-49af-429b-a9aa-b4694a3d5c4a None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:48:29.148491 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cacb2fa7-49af-429b-a9aa-b4694a3d5c4a None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:48:29.148771 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cacb2fa7-49af-429b-a9aa-b4694a3d5c4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:48:29.190645 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-cacb2fa7-49af-429b-a9aa-b4694a3d5c4a None None] MFA Rules not processed for user `1cd482a61e8a420db45bb532e5dae6e2`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:48:29.191976 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-cacb2fa7-49af-429b-a9aa-b4694a3d5c4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:48:29.209711 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e6a322-c7ac-4a2b-a4dc-224f77a9e688 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:48:29.210474 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e6a322-c7ac-4a2b-a4dc-224f77a9e688 admin admin] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:48:29.210765 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e6a322-c7ac-4a2b-a4dc-224f77a9e688 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:48:29.211046 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e6a322-c7ac-4a2b-a4dc-224f77a9e688 admin admin] PATH_INFO: `/v3/users` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:48:29.211769 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0e6a322-c7ac-4a2b-a4dc-224f77a9e688 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:48:29.214015 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0e6a322-c7ac-4a2b-a4dc-224f77a9e688 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:48:29.238776 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1409/2813] 10.222.0.33 () {60 vars in 1143 bytes} [Mon Jul 25 05:48:29 2022] POST /identity/v3/users => generated 286 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 05:48:29.243190 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e6a322-c7ac-4a2b-a4dc-224f77a9e688 admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:48:29.254515 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e6a322-c7ac-4a2b-a4dc-224f77a9e688 admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:48:29.266380 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a14b93a9-4676-43de-a8ba-6f3fb8df2411 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:48:29.266380 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14b93a9-4676-43de-a8ba-6f3fb8df2411 admin admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:48:29.266380 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14b93a9-4676-43de-a8ba-6f3fb8df2411 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:48:29.266380 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14b93a9-4676-43de-a8ba-6f3fb8df2411 admin admin] PATH_INFO: `/v3/roles` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:48:29.266874 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a14b93a9-4676-43de-a8ba-6f3fb8df2411 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:48:29.266874 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a14b93a9-4676-43de-a8ba-6f3fb8df2411 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:48:29.269727 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1405/2814] 10.222.0.33 () {58 vars in 1018 bytes} [Mon Jul 25 05:48:29 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 05:48:29.278794 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1410/2815] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jul 25 05:48:29 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 25 05:48:29.280380 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a14b93a9-4676-43de-a8ba-6f3fb8df2411 admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:48:29.280508 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cacb2fa7-49af-429b-a9aa-b4694a3d5c4a None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:48:29.282186 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-a14b93a9-4676-43de-a8ba-6f3fb8df2411 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:48:29.291969 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cacb2fa7-49af-429b-a9aa-b4694a3d5c4a None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:48:29.296944 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e83ec336-688f-490e-872f-cf4a012dcef9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:48:29.297612 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83ec336-688f-490e-872f-cf4a012dcef9 admin admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:48:29.297900 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83ec336-688f-490e-872f-cf4a012dcef9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:48:29.298178 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83ec336-688f-490e-872f-cf4a012dcef9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:48:29.298972 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e83ec336-688f-490e-872f-cf4a012dcef9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:48:29.300561 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e83ec336-688f-490e-872f-cf4a012dcef9 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:48:29.311412 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1406/2816] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jul 25 05:48:29 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 05:48:29.315552 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e83ec336-688f-490e-872f-cf4a012dcef9 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:48:29.325979 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e83ec336-688f-490e-872f-cf4a012dcef9 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:48:29.330999 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d387550-c5e0-454e-838e-414a2596d7a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:48:29.331860 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d387550-c5e0-454e-838e-414a2596d7a3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:48:29.332137 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d387550-c5e0-454e-838e-414a2596d7a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:48:29.332409 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d387550-c5e0-454e-838e-414a2596d7a3 admin admin] PATH_INFO: `/v3/projects/360d1ad816bb474cb4195d3e531097de/users/1ef5eb710ded44a28703a68810423fd7/roles/5074e74e35f54d4b9bb415c34999a82a` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:48:29.343449 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a14b93a9-4676-43de-a8ba-6f3fb8df2411 admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:48:29.347142 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6112232-3885-416f-a669-eb51f648a9e7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b2931200f99e43d79a50a9f6e02738e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92afba266cce4d1bad4134c351e74be3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:48:29.347404 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d387550-c5e0-454e-838e-414a2596d7a3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=360d1ad816bb474cb4195d3e531097de, user_id=1ef5eb710ded44a28703a68810423fd7, role_id=5074e74e35f54d4b9bb415c34999a82a)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:48:29.347791 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6112232-3885-416f-a669-eb51f648a9e7 service neutron] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:48:29.348078 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6112232-3885-416f-a669-eb51f648a9e7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:48:29.348355 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6112232-3885-416f-a669-eb51f648a9e7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:48:29.350822 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d387550-c5e0-454e-838e-414a2596d7a3 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:48:29.352967 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:48:29.360314 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6112232-3885-416f-a669-eb51f648a9e7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:48:29.362037 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6112232-3885-416f-a669-eb51f648a9e7 service neutron] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:48:29.366684 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1407/2817] 10.222.0.33 () {60 vars in 1482 bytes} [Mon Jul 25 05:48:29 2022] PUT /identity/v3/projects/360d1ad816bb474cb4195d3e531097de/users/1ef5eb710ded44a28703a68810423fd7/roles/5074e74e35f54d4b9bb415c34999a82a => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 25 05:48:29.374971 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1411/2818] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jul 25 05:48:29 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 05:48:29.381651 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32dfe374-8cf5-4660-8cf8-ea5abaa0bd7a None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:48:29.382048 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32dfe374-8cf5-4660-8cf8-ea5abaa0bd7a None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:48:29.382378 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32dfe374-8cf5-4660-8cf8-ea5abaa0bd7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:48:29.410876 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-32dfe374-8cf5-4660-8cf8-ea5abaa0bd7a None None] MFA Rules not processed for user `1ef5eb710ded44a28703a68810423fd7`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:48:29.412640 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-32dfe374-8cf5-4660-8cf8-ea5abaa0bd7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:48:29.437698 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6112232-3885-416f-a669-eb51f648a9e7 service neutron] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:48:29.439132 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-a6112232-3885-416f-a669-eb51f648a9e7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:48:29.481314 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1408/2819] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 25 05:48:29 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:48:29.486026 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32dfe374-8cf5-4660-8cf8-ea5abaa0bd7a None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:48:29.495162 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:48:29.512037 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6112232-3885-416f-a669-eb51f648a9e7 service neutron] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:48:29.547852 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32dfe374-8cf5-4660-8cf8-ea5abaa0bd7a None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:48:29.552437 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ea06b2-0a0d-4d55-994b-3d06396e7983 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:48:29.553711 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ea06b2-0a0d-4d55-994b-3d06396e7983 admin admin] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:48:29.553995 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ea06b2-0a0d-4d55-994b-3d06396e7983 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:48:29.554275 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ea06b2-0a0d-4d55-994b-3d06396e7983 admin admin] PATH_INFO: `/v3/projects` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:48:29.554966 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3ea06b2-0a0d-4d55-994b-3d06396e7983 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:48:29.556889 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3ea06b2-0a0d-4d55-994b-3d06396e7983 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:48:29.574473 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddbb7aa4-9dfb-4913-9bd7-fe92a3a1d449 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '317790016e7047fe96af73925de5d872', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92afba266cce4d1bad4134c351e74be3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:48:29.575193 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddbb7aa4-9dfb-4913-9bd7-fe92a3a1d449 service placement] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:48:29.575546 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddbb7aa4-9dfb-4913-9bd7-fe92a3a1d449 service placement] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:48:29.575864 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddbb7aa4-9dfb-4913-9bd7-fe92a3a1d449 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:48:29.577739 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-ddbb7aa4-9dfb-4913-9bd7-fe92a3a1d449 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:48:29.582727 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1409/2820] 10.222.0.33 () {60 vars in 1152 bytes} [Mon Jul 25 05:48:29 2022] POST /identity/v3/projects => generated 371 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 25 05:48:29.586627 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ea06b2-0a0d-4d55-994b-3d06396e7983 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:48:29.589826 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddbb7aa4-9dfb-4913-9bd7-fe92a3a1d449 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:48:29.591479 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddbb7aa4-9dfb-4913-9bd7-fe92a3a1d449 service placement] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:48:29.603114 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1412/2821] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jul 25 05:48:29 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 166 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 25 05:48:29.650472 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ea06b2-0a0d-4d55-994b-3d06396e7983 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:48:29.655230 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f66d932a-d890-4365-bb56-f414e7c8bbb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:48:29.655904 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66d932a-d890-4365-bb56-f414e7c8bbb0 admin admin] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:48:29.656186 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66d932a-d890-4365-bb56-f414e7c8bbb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:48:29.656747 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66d932a-d890-4365-bb56-f414e7c8bbb0 admin admin] PATH_INFO: `/v3/users` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:48:29.657479 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f66d932a-d890-4365-bb56-f414e7c8bbb0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:48:29.659384 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f66d932a-d890-4365-bb56-f414e7c8bbb0 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:48:29.674327 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:INSERT=5,keystone:SELECT=95 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:48:29.682591 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1410/2822] 10.222.0.33 () {60 vars in 1143 bytes} [Mon Jul 25 05:48:29 2022] POST /identity/v3/users => generated 292 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 05:48:29.686594 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddbb7aa4-9dfb-4913-9bd7-fe92a3a1d449 service placement] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:48:29.703129 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddbb7aa4-9dfb-4913-9bd7-fe92a3a1d449 service placement] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:48:29.708388 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8439fd1-6a12-4f23-b1fe-a7b2925de266 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:48:29.709238 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8439fd1-6a12-4f23-b1fe-a7b2925de266 admin admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:48:29.709531 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8439fd1-6a12-4f23-b1fe-a7b2925de266 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:48:29.709817 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8439fd1-6a12-4f23-b1fe-a7b2925de266 admin admin] PATH_INFO: `/v3/roles` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:48:29.710559 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8439fd1-6a12-4f23-b1fe-a7b2925de266 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:48:29.712158 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8439fd1-6a12-4f23-b1fe-a7b2925de266 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:48:29.722422 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1413/2823] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jul 25 05:48:29 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 05:48:29.726796 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f66d932a-d890-4365-bb56-f414e7c8bbb0 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:48:29.737122 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f66d932a-d890-4365-bb56-f414e7c8bbb0 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:48:29.742354 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f25cc231-dd81-49c3-a4c6-9515028919b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:48:29.743258 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25cc231-dd81-49c3-a4c6-9515028919b9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:48:29.743542 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25cc231-dd81-49c3-a4c6-9515028919b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:48:29.743821 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25cc231-dd81-49c3-a4c6-9515028919b9 admin admin] PATH_INFO: `/v3/projects/f019c4d22dbb4174a24818c2001d3844/users/cf9fa52de3d94c219a7b019ecaeb8ee3/roles/39e5c666285a42fe8fc2c3afca6260d1` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:48:29.758959 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f25cc231-dd81-49c3-a4c6-9515028919b9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f019c4d22dbb4174a24818c2001d3844, user_id=cf9fa52de3d94c219a7b019ecaeb8ee3, role_id=39e5c666285a42fe8fc2c3afca6260d1)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:48:29.761617 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f25cc231-dd81-49c3-a4c6-9515028919b9 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:48:29.774316 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1411/2824] 10.222.0.33 () {60 vars in 1482 bytes} [Mon Jul 25 05:48:29 2022] PUT /identity/v3/projects/f019c4d22dbb4174a24818c2001d3844/users/cf9fa52de3d94c219a7b019ecaeb8ee3/roles/39e5c666285a42fe8fc2c3afca6260d1 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 05:48:29.778404 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8439fd1-6a12-4f23-b1fe-a7b2925de266 admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:48:29.788200 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8439fd1-6a12-4f23-b1fe-a7b2925de266 admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:48:29.845339 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e705e7e6-ed22-4ee1-9ac9-c31e0b2ade7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:48:29.846008 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e705e7e6-ed22-4ee1-9ac9-c31e0b2ade7f admin admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:48:29.846295 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e705e7e6-ed22-4ee1-9ac9-c31e0b2ade7f admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:48:29.846573 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e705e7e6-ed22-4ee1-9ac9-c31e0b2ade7f admin admin] PATH_INFO: `/v3/roles` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:48:29.847302 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e705e7e6-ed22-4ee1-9ac9-c31e0b2ade7f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:48:29.848926 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e705e7e6-ed22-4ee1-9ac9-c31e0b2ade7f admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:48:29.859554 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1414/2825] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jul 25 05:48:29 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 05:48:29.863863 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f25cc231-dd81-49c3-a4c6-9515028919b9 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:48:29.873969 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f25cc231-dd81-49c3-a4c6-9515028919b9 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:48:29.878586 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be3f0c0-fdc6-466c-8ebb-e132b4701389 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:48:29.879458 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be3f0c0-fdc6-466c-8ebb-e132b4701389 admin admin] REQUEST_METHOD: `PUT` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:48:29.879741 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be3f0c0-fdc6-466c-8ebb-e132b4701389 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:48:29.880018 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be3f0c0-fdc6-466c-8ebb-e132b4701389 admin admin] PATH_INFO: `/v3/projects/f019c4d22dbb4174a24818c2001d3844/users/cf9fa52de3d94c219a7b019ecaeb8ee3/roles/39e5c666285a42fe8fc2c3afca6260d1` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:48:29.882030 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8be3f0c0-fdc6-466c-8ebb-e132b4701389 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f019c4d22dbb4174a24818c2001d3844, user_id=cf9fa52de3d94c219a7b019ecaeb8ee3, role_id=39e5c666285a42fe8fc2c3afca6260d1)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:48:29.884641 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8be3f0c0-fdc6-466c-8ebb-e132b4701389 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:48:29.894675 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1412/2826] 10.222.0.33 () {60 vars in 1482 bytes} [Mon Jul 25 05:48:29 2022] PUT /identity/v3/projects/f019c4d22dbb4174a24818c2001d3844/users/cf9fa52de3d94c219a7b019ecaeb8ee3/roles/39e5c666285a42fe8fc2c3afca6260d1 => generated 0 bytes in 31 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 05:48:29.906275 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8490438b-e212-4673-9899-57d63599caf5 None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:48:29.906806 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8490438b-e212-4673-9899-57d63599caf5 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:48:29.906908 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8490438b-e212-4673-9899-57d63599caf5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:48:29.934252 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-8490438b-e212-4673-9899-57d63599caf5 None None] MFA Rules not processed for user `cf9fa52de3d94c219a7b019ecaeb8ee3`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:48:29.935462 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-8490438b-e212-4673-9899-57d63599caf5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:48:29.995089 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:48:30.004416 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1415/2827] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 25 05:48:29 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:48:30.015951 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be3f0c0-fdc6-466c-8ebb-e132b4701389 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:48:30.017828 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-8be3f0c0-fdc6-466c-8ebb-e132b4701389 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:48:30.075443 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be3f0c0-fdc6-466c-8ebb-e132b4701389 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:48:30.079056 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d4bab1f-3a22-4dd9-89b8-20806eac34f0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db57b478c25643dbaf936f90f6e17d0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92afba266cce4d1bad4134c351e74be3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:48:30.079700 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4bab1f-3a22-4dd9-89b8-20806eac34f0 service cinder] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:48:30.079993 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4bab1f-3a22-4dd9-89b8-20806eac34f0 service cinder] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:48:30.080273 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4bab1f-3a22-4dd9-89b8-20806eac34f0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:48:30.089435 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d4bab1f-3a22-4dd9-89b8-20806eac34f0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:48:30.090859 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d4bab1f-3a22-4dd9-89b8-20806eac34f0 service cinder] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:48:30.101062 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1413/2828] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jul 25 05:48:30 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 05:48:30.171927 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8490438b-e212-4673-9899-57d63599caf5 None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:48:30.182858 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8490438b-e212-4673-9899-57d63599caf5 None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:48:30.186532 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf441cb-038e-42d0-bda7-b31bd047342e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db57b478c25643dbaf936f90f6e17d0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92afba266cce4d1bad4134c351e74be3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:48:30.187169 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf441cb-038e-42d0-bda7-b31bd047342e service cinder] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:48:30.187455 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf441cb-038e-42d0-bda7-b31bd047342e service cinder] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:48:30.187732 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf441cb-038e-42d0-bda7-b31bd047342e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:48:30.243435 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cf441cb-038e-42d0-bda7-b31bd047342e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:48:30.244906 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cf441cb-038e-42d0-bda7-b31bd047342e service cinder] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:48:30.253760 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1416/2829] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jul 25 05:48:30 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 05:48:40.096063 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=42,keystone:INSERT=2 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:48:40.249681 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=30 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:49:09.490330 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d4bab1f-3a22-4dd9-89b8-20806eac34f0 service cinder] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:49:09.553287 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d4bab1f-3a22-4dd9-89b8-20806eac34f0 service cinder] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:49:09.557753 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8360bca1-4d34-489c-b29d-ed58d9a3eba3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:49:09.558622 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8360bca1-4d34-489c-b29d-ed58d9a3eba3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:49:09.558907 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8360bca1-4d34-489c-b29d-ed58d9a3eba3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:49:09.559185 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8360bca1-4d34-489c-b29d-ed58d9a3eba3 admin admin] PATH_INFO: `/v3/users/1ef5eb710ded44a28703a68810423fd7` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:49:09.560360 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8360bca1-4d34-489c-b29d-ed58d9a3eba3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1ef5eb710ded44a28703a68810423fd7)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:49:09.562301 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8360bca1-4d34-489c-b29d-ed58d9a3eba3 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:49:09.608648 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-8360bca1-4d34-489c-b29d-ed58d9a3eba3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1ef5eb710ded44a28703a68810423fd7'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:49:09.609382 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-8360bca1-4d34-489c-b29d-ed58d9a3eba3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1ef5eb710ded44a28703a68810423fd7'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:49:09.616486 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-8360bca1-4d34-489c-b29d-ed58d9a3eba3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1ef5eb710ded44a28703a68810423fd7'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:49:09.628131 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-8360bca1-4d34-489c-b29d-ed58d9a3eba3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1ef5eb710ded44a28703a68810423fd7'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:49:09.628770 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-8360bca1-4d34-489c-b29d-ed58d9a3eba3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1ef5eb710ded44a28703a68810423fd7'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:49:09.641937 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1414/2830] 10.222.0.33 () {58 vars in 1223 bytes} [Mon Jul 25 05:49:09 2022] DELETE /identity/v3/users/1ef5eb710ded44a28703a68810423fd7 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 05:49:09.646406 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8bab53-05d0-4766-b9bf-ae38a9da01b4 None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:49:09.646907 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8bab53-05d0-4766-b9bf-ae38a9da01b4 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:49:09.647027 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8bab53-05d0-4766-b9bf-ae38a9da01b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:49:09.664432 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-2c8bab53-05d0-4766-b9bf-ae38a9da01b4 None None] MFA Rules not processed for user `1eae540242d94c10b0ad33960a855f08`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:49:09.665661 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-2c8bab53-05d0-4766-b9bf-ae38a9da01b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:49:09.730353 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1417/2831] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 25 05:49:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 05:49:09.740128 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8360bca1-4d34-489c-b29d-ed58d9a3eba3 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:49:09.741581 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-8360bca1-4d34-489c-b29d-ed58d9a3eba3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:49:09.799216 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8360bca1-4d34-489c-b29d-ed58d9a3eba3 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:49:09.802911 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-235176c5-0280-49c7-acb2-9b88e5206855 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b2931200f99e43d79a50a9f6e02738e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92afba266cce4d1bad4134c351e74be3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:49:09.803572 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235176c5-0280-49c7-acb2-9b88e5206855 service neutron] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:49:09.803859 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235176c5-0280-49c7-acb2-9b88e5206855 service neutron] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:49:09.804141 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235176c5-0280-49c7-acb2-9b88e5206855 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:49:09.814185 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-235176c5-0280-49c7-acb2-9b88e5206855 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:49:09.815599 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-235176c5-0280-49c7-acb2-9b88e5206855 service neutron] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:49:09.825040 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1415/2832] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jul 25 05:49:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 05:49:10.086223 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c8bab53-05d0-4766-b9bf-ae38a9da01b4 None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:49:10.087749 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-2c8bab53-05d0-4766-b9bf-ae38a9da01b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:49:10.101200 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c8bab53-05d0-4766-b9bf-ae38a9da01b4 None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:49:10.105883 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-265af741-b937-4286-9cde-a08420d79517 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:49:10.106753 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265af741-b937-4286-9cde-a08420d79517 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:49:10.107044 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265af741-b937-4286-9cde-a08420d79517 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:49:10.107325 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265af741-b937-4286-9cde-a08420d79517 admin admin] PATH_INFO: `/v3/projects/360d1ad816bb474cb4195d3e531097de` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:49:10.108573 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-265af741-b937-4286-9cde-a08420d79517 admin admin] RBAC: Authorizing `identity:delete_project(project_id=360d1ad816bb474cb4195d3e531097de)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:49:10.111318 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-265af741-b937-4286-9cde-a08420d79517 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:49:10.158691 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-265af741-b937-4286-9cde-a08420d79517 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '360d1ad816bb474cb4195d3e531097de', 'request_id': 'req-265af741-b937-4286-9cde-a08420d79517'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:49:10.164166 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-265af741-b937-4286-9cde-a08420d79517 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '360d1ad816bb474cb4195d3e531097de', 'request_id': 'req-265af741-b937-4286-9cde-a08420d79517'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:49:10.170559 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-265af741-b937-4286-9cde-a08420d79517 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '360d1ad816bb474cb4195d3e531097de', 'request_id': 'req-265af741-b937-4286-9cde-a08420d79517'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:49:10.183003 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-265af741-b937-4286-9cde-a08420d79517 admin admin] The token cache is being invalidate because project 360d1ad816bb474cb4195d3e531097de was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80248) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 05:49:10.183296 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-265af741-b937-4286-9cde-a08420d79517 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:49:10.183888 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-265af741-b937-4286-9cde-a08420d79517 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:49:10.185465 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1418/2833] 10.222.0.33 () {58 vars in 1232 bytes} [Mon Jul 25 05:49:10 2022] DELETE /identity/v3/projects/360d1ad816bb474cb4195d3e531097de => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 05:49:10.190223 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-235176c5-0280-49c7-acb2-9b88e5206855 service neutron] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:49:10.199783 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-235176c5-0280-49c7-acb2-9b88e5206855 service neutron] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:49:10.217760 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:49:10.257771 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b39fac2-6792-45ea-a7a5-1b9141456dbb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:49:10.258641 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b39fac2-6792-45ea-a7a5-1b9141456dbb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:49:10.258929 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b39fac2-6792-45ea-a7a5-1b9141456dbb admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:49:10.259208 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b39fac2-6792-45ea-a7a5-1b9141456dbb admin admin] PATH_INFO: `/v3/users/cf9fa52de3d94c219a7b019ecaeb8ee3` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:49:10.260400 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b39fac2-6792-45ea-a7a5-1b9141456dbb admin admin] RBAC: Authorizing `identity:delete_user(user_id=cf9fa52de3d94c219a7b019ecaeb8ee3)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:49:10.262358 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b39fac2-6792-45ea-a7a5-1b9141456dbb admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:49:10.307466 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-2b39fac2-6792-45ea-a7a5-1b9141456dbb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cf9fa52de3d94c219a7b019ecaeb8ee3'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:49:10.308103 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-2b39fac2-6792-45ea-a7a5-1b9141456dbb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cf9fa52de3d94c219a7b019ecaeb8ee3'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:49:10.315087 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-2b39fac2-6792-45ea-a7a5-1b9141456dbb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cf9fa52de3d94c219a7b019ecaeb8ee3'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:49:10.326656 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-2b39fac2-6792-45ea-a7a5-1b9141456dbb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cf9fa52de3d94c219a7b019ecaeb8ee3'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:49:10.327298 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-2b39fac2-6792-45ea-a7a5-1b9141456dbb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cf9fa52de3d94c219a7b019ecaeb8ee3'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:49:10.340660 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1416/2834] 10.222.0.33 () {58 vars in 1223 bytes} [Mon Jul 25 05:49:10 2022] DELETE /identity/v3/users/cf9fa52de3d94c219a7b019ecaeb8ee3 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 05:49:10.617435 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-265af741-b937-4286-9cde-a08420d79517 admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:49:10.618894 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-265af741-b937-4286-9cde-a08420d79517 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:49:10.633131 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-265af741-b937-4286-9cde-a08420d79517 admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:49:10.690946 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-493e2111-c9a5-4a90-b1f2-0035522f50ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:49:10.691735 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493e2111-c9a5-4a90-b1f2-0035522f50ff admin admin] REQUEST_METHOD: `DELETE` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:49:10.692023 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493e2111-c9a5-4a90-b1f2-0035522f50ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:49:10.692302 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493e2111-c9a5-4a90-b1f2-0035522f50ff admin admin] PATH_INFO: `/v3/projects/f019c4d22dbb4174a24818c2001d3844` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:49:10.693503 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-493e2111-c9a5-4a90-b1f2-0035522f50ff admin admin] RBAC: Authorizing `identity:delete_project(project_id=f019c4d22dbb4174a24818c2001d3844)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:49:10.695392 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-493e2111-c9a5-4a90-b1f2-0035522f50ff admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:49:10.726269 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:49:10.743784 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-493e2111-c9a5-4a90-b1f2-0035522f50ff admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f019c4d22dbb4174a24818c2001d3844', 'request_id': 'req-493e2111-c9a5-4a90-b1f2-0035522f50ff'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:49:10.749514 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-493e2111-c9a5-4a90-b1f2-0035522f50ff admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f019c4d22dbb4174a24818c2001d3844', 'request_id': 'req-493e2111-c9a5-4a90-b1f2-0035522f50ff'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:49:10.756154 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-493e2111-c9a5-4a90-b1f2-0035522f50ff admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f019c4d22dbb4174a24818c2001d3844', 'request_id': 'req-493e2111-c9a5-4a90-b1f2-0035522f50ff'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:49:10.769241 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-493e2111-c9a5-4a90-b1f2-0035522f50ff admin admin] The token cache is being invalidate because project f019c4d22dbb4174a24818c2001d3844 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80248) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 05:49:10.769529 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-493e2111-c9a5-4a90-b1f2-0035522f50ff admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:49:10.770149 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-493e2111-c9a5-4a90-b1f2-0035522f50ff admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:49:10.771759 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1419/2835] 10.222.0.33 () {58 vars in 1232 bytes} [Mon Jul 25 05:49:10 2022] DELETE /identity/v3/projects/f019c4d22dbb4174a24818c2001d3844 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 05:49:10.778935 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7327d937-5b0d-460f-969b-49c58266bd8d None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:49:10.779447 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7327d937-5b0d-460f-969b-49c58266bd8d None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:49:10.779562 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7327d937-5b0d-460f-969b-49c58266bd8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:49:10.798560 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-7327d937-5b0d-460f-969b-49c58266bd8d None None] MFA Rules not processed for user `1eae540242d94c10b0ad33960a855f08`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:49:10.799790 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-7327d937-5b0d-460f-969b-49c58266bd8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:49:10.858618 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1417/2836] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 25 05:49:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:49:10.867182 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23bfa318-856d-4773-b6b8-a09036a0f6e4 None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:49:10.867483 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23bfa318-856d-4773-b6b8-a09036a0f6e4 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:49:10.867771 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23bfa318-856d-4773-b6b8-a09036a0f6e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:49:10.886724 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-23bfa318-856d-4773-b6b8-a09036a0f6e4 None None] MFA Rules not processed for user `1eae540242d94c10b0ad33960a855f08`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:49:10.887952 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-23bfa318-856d-4773-b6b8-a09036a0f6e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:49:10.896177 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1420/2837] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 25 05:49:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:49:10.900298 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7327d937-5b0d-460f-969b-49c58266bd8d None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:49:10.910022 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7327d937-5b0d-460f-969b-49c58266bd8d None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:49:10.914488 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122ecf7c-df10-40fa-bc57-a460eba448ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:49:10.915134 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122ecf7c-df10-40fa-bc57-a460eba448ec admin admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:49:10.915427 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122ecf7c-df10-40fa-bc57-a460eba448ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:49:10.915700 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122ecf7c-df10-40fa-bc57-a460eba448ec admin admin] PATH_INFO: `/v3/domains` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:49:10.916420 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-122ecf7c-df10-40fa-bc57-a460eba448ec admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:49:10.916725 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-122ecf7c-df10-40fa-bc57-a460eba448ec admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:49:10.918333 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-122ecf7c-df10-40fa-bc57-a460eba448ec admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:49:10.930318 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1418/2838] 10.222.0.33 () {58 vars in 1152 bytes} [Mon Jul 25 05:49:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:49:10.934476 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23bfa318-856d-4773-b6b8-a09036a0f6e4 None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:49:10.944348 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23bfa318-856d-4773-b6b8-a09036a0f6e4 None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:49:10.949015 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca874793-15e8-41a9-8389-989a7aa94d58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:49:10.949684 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca874793-15e8-41a9-8389-989a7aa94d58 admin admin] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:49:10.949953 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca874793-15e8-41a9-8389-989a7aa94d58 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:49:10.950231 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca874793-15e8-41a9-8389-989a7aa94d58 admin admin] PATH_INFO: `/v3/projects` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:49:10.950927 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca874793-15e8-41a9-8389-989a7aa94d58 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:49:10.952856 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca874793-15e8-41a9-8389-989a7aa94d58 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:49:10.978405 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1421/2839] 10.222.0.33 () {60 vars in 1152 bytes} [Mon Jul 25 05:49:10 2022] POST /identity/v3/projects => generated 379 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 05:49:10.982542 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122ecf7c-df10-40fa-bc57-a460eba448ec admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:49:10.992979 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122ecf7c-df10-40fa-bc57-a460eba448ec admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:49:11.019027 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:49:11.052860 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f800e774-ce2f-4119-9318-7e0ad34fd513 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:49:11.053528 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f800e774-ce2f-4119-9318-7e0ad34fd513 admin admin] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:49:11.053827 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f800e774-ce2f-4119-9318-7e0ad34fd513 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:49:11.054082 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f800e774-ce2f-4119-9318-7e0ad34fd513 admin admin] PATH_INFO: `/v3/users` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:49:11.054781 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f800e774-ce2f-4119-9318-7e0ad34fd513 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:49:11.056708 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f800e774-ce2f-4119-9318-7e0ad34fd513 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:49:11.076407 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1419/2840] 10.222.0.33 () {60 vars in 1143 bytes} [Mon Jul 25 05:49:10 2022] POST /identity/v3/users => generated 290 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 05:49:11.080384 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca874793-15e8-41a9-8389-989a7aa94d58 admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:49:11.090530 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca874793-15e8-41a9-8389-989a7aa94d58 admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:49:11.095147 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad1b8e8-a3d8-4d27-936e-5781cfd40333 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:49:11.095913 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad1b8e8-a3d8-4d27-936e-5781cfd40333 admin admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:49:11.096335 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad1b8e8-a3d8-4d27-936e-5781cfd40333 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:49:11.096620 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad1b8e8-a3d8-4d27-936e-5781cfd40333 admin admin] PATH_INFO: `/v3/roles` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:49:11.097387 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ad1b8e8-a3d8-4d27-936e-5781cfd40333 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:49:11.098970 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ad1b8e8-a3d8-4d27-936e-5781cfd40333 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:49:11.110573 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1422/2841] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jul 25 05:49:11 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 05:49:11.115121 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f800e774-ce2f-4119-9318-7e0ad34fd513 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:49:11.125003 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f800e774-ce2f-4119-9318-7e0ad34fd513 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:49:11.130131 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff2f1f84-92fd-492e-97cf-cab297d8af97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:49:11.130785 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2f1f84-92fd-492e-97cf-cab297d8af97 admin admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:49:11.131068 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2f1f84-92fd-492e-97cf-cab297d8af97 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:49:11.131344 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2f1f84-92fd-492e-97cf-cab297d8af97 admin admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:49:11.132067 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff2f1f84-92fd-492e-97cf-cab297d8af97 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:49:11.133676 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff2f1f84-92fd-492e-97cf-cab297d8af97 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:49:11.143404 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1420/2842] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jul 25 05:49:11 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 25 05:49:11.147470 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad1b8e8-a3d8-4d27-936e-5781cfd40333 admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:49:11.157323 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad1b8e8-a3d8-4d27-936e-5781cfd40333 admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:49:11.162630 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-578055b1-5b35-4e5a-a24a-01eef66528fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:49:11.163500 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578055b1-5b35-4e5a-a24a-01eef66528fc admin admin] REQUEST_METHOD: `PUT` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:49:11.163785 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578055b1-5b35-4e5a-a24a-01eef66528fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:49:11.164060 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578055b1-5b35-4e5a-a24a-01eef66528fc admin admin] PATH_INFO: `/v3/projects/1755f8a133a6431494f0fe81d5a4cf7f/users/d59eec85f359439a920d84494a70906d/roles/5074e74e35f54d4b9bb415c34999a82a` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:49:11.179204 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-578055b1-5b35-4e5a-a24a-01eef66528fc admin admin] RBAC: Authorizing `identity:create_grant(project_id=1755f8a133a6431494f0fe81d5a4cf7f, user_id=d59eec85f359439a920d84494a70906d, role_id=5074e74e35f54d4b9bb415c34999a82a)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:49:11.181904 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-578055b1-5b35-4e5a-a24a-01eef66528fc admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:49:11.195618 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1423/2843] 10.222.0.33 () {60 vars in 1482 bytes} [Mon Jul 25 05:49:11 2022] PUT /identity/v3/projects/1755f8a133a6431494f0fe81d5a4cf7f/users/d59eec85f359439a920d84494a70906d/roles/5074e74e35f54d4b9bb415c34999a82a => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 05:49:11.206721 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad9e5df-9aad-4a2e-a7ea-8763aae5a34c None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:49:11.207019 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad9e5df-9aad-4a2e-a7ea-8763aae5a34c None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:49:11.207332 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad9e5df-9aad-4a2e-a7ea-8763aae5a34c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:49:11.233742 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-7ad9e5df-9aad-4a2e-a7ea-8763aae5a34c None None] MFA Rules not processed for user `d59eec85f359439a920d84494a70906d`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:49:11.235055 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-7ad9e5df-9aad-4a2e-a7ea-8763aae5a34c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:49:11.298598 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1421/2844] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 25 05:49:11 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:49:11.303598 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-578055b1-5b35-4e5a-a24a-01eef66528fc admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:49:11.370124 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-578055b1-5b35-4e5a-a24a-01eef66528fc admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:49:11.374727 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1394b000-9505-444f-b1e6-a8614305b639 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:49:11.375373 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1394b000-9505-444f-b1e6-a8614305b639 admin admin] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:49:11.375656 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1394b000-9505-444f-b1e6-a8614305b639 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:49:11.375928 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1394b000-9505-444f-b1e6-a8614305b639 admin admin] PATH_INFO: `/v3/projects` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:49:11.376612 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1394b000-9505-444f-b1e6-a8614305b639 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:49:11.379079 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1394b000-9505-444f-b1e6-a8614305b639 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:49:11.403325 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1424/2845] 10.222.0.33 () {60 vars in 1152 bytes} [Mon Jul 25 05:49:11 2022] POST /identity/v3/projects => generated 381 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 05:49:11.407382 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad9e5df-9aad-4a2e-a7ea-8763aae5a34c None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:49:11.418569 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad9e5df-9aad-4a2e-a7ea-8763aae5a34c None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:49:11.473669 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-555e4115-f531-45ec-bbc9-3b4f6be736be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:49:11.474262 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555e4115-f531-45ec-bbc9-3b4f6be736be admin admin] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:49:11.474550 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555e4115-f531-45ec-bbc9-3b4f6be736be admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:49:11.475084 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555e4115-f531-45ec-bbc9-3b4f6be736be admin admin] PATH_INFO: `/v3/users` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:49:11.475791 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-555e4115-f531-45ec-bbc9-3b4f6be736be admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:49:11.477765 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-555e4115-f531-45ec-bbc9-3b4f6be736be admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:49:11.497514 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1422/2846] 10.222.0.33 () {60 vars in 1143 bytes} [Mon Jul 25 05:49:11 2022] POST /identity/v3/users => generated 297 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 05:49:11.501573 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1394b000-9505-444f-b1e6-a8614305b639 admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:49:11.511870 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1394b000-9505-444f-b1e6-a8614305b639 admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:49:11.516462 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e21ed95b-eadf-4477-819f-e3f4bda2e4ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:49:11.517141 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21ed95b-eadf-4477-819f-e3f4bda2e4ae admin admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:49:11.517463 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21ed95b-eadf-4477-819f-e3f4bda2e4ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:49:11.517881 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21ed95b-eadf-4477-819f-e3f4bda2e4ae admin admin] PATH_INFO: `/v3/roles` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:49:11.518608 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e21ed95b-eadf-4477-819f-e3f4bda2e4ae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:49:11.520188 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e21ed95b-eadf-4477-819f-e3f4bda2e4ae admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:49:11.530495 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1425/2847] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jul 25 05:49:11 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 05:49:11.535006 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-555e4115-f531-45ec-bbc9-3b4f6be736be admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:49:11.545444 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-555e4115-f531-45ec-bbc9-3b4f6be736be admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:49:11.550856 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84737d15-5bc8-410d-8b8f-0ce8d51a8a41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:49:11.551737 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84737d15-5bc8-410d-8b8f-0ce8d51a8a41 admin admin] REQUEST_METHOD: `PUT` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:49:11.552022 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84737d15-5bc8-410d-8b8f-0ce8d51a8a41 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:49:11.552418 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84737d15-5bc8-410d-8b8f-0ce8d51a8a41 admin admin] PATH_INFO: `/v3/projects/93d502514f39421c8d8ce22989785c76/users/09903e8bf45e4839b5fcd01b272e4813/roles/39e5c666285a42fe8fc2c3afca6260d1` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:49:11.567960 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84737d15-5bc8-410d-8b8f-0ce8d51a8a41 admin admin] RBAC: Authorizing `identity:create_grant(project_id=93d502514f39421c8d8ce22989785c76, user_id=09903e8bf45e4839b5fcd01b272e4813, role_id=39e5c666285a42fe8fc2c3afca6260d1)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:49:11.570620 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84737d15-5bc8-410d-8b8f-0ce8d51a8a41 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:49:11.583877 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1423/2848] 10.222.0.33 () {60 vars in 1482 bytes} [Mon Jul 25 05:49:11 2022] PUT /identity/v3/projects/93d502514f39421c8d8ce22989785c76/users/09903e8bf45e4839b5fcd01b272e4813/roles/39e5c666285a42fe8fc2c3afca6260d1 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 05:49:11.587945 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e21ed95b-eadf-4477-819f-e3f4bda2e4ae admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:49:11.598776 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e21ed95b-eadf-4477-819f-e3f4bda2e4ae admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:49:11.618251 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:49:11.656281 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf162f3c-533b-41c1-9d1c-e14e5b56ffe5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:49:11.656986 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf162f3c-533b-41c1-9d1c-e14e5b56ffe5 admin admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:49:11.657272 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf162f3c-533b-41c1-9d1c-e14e5b56ffe5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:49:11.657552 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf162f3c-533b-41c1-9d1c-e14e5b56ffe5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:49:11.659080 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf162f3c-533b-41c1-9d1c-e14e5b56ffe5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:49:11.660715 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf162f3c-533b-41c1-9d1c-e14e5b56ffe5 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:49:11.670530 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1426/2849] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jul 25 05:49:11 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 05:49:11.674882 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84737d15-5bc8-410d-8b8f-0ce8d51a8a41 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:49:11.680307 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:49:11.687366 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84737d15-5bc8-410d-8b8f-0ce8d51a8a41 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:49:11.692399 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bde2bd4-b6be-4fc5-a494-a64f98fe3902 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:49:11.693395 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bde2bd4-b6be-4fc5-a494-a64f98fe3902 admin admin] REQUEST_METHOD: `PUT` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:49:11.693676 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bde2bd4-b6be-4fc5-a494-a64f98fe3902 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:49:11.693947 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bde2bd4-b6be-4fc5-a494-a64f98fe3902 admin admin] PATH_INFO: `/v3/projects/93d502514f39421c8d8ce22989785c76/users/09903e8bf45e4839b5fcd01b272e4813/roles/39e5c666285a42fe8fc2c3afca6260d1` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:49:11.695912 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bde2bd4-b6be-4fc5-a494-a64f98fe3902 admin admin] RBAC: Authorizing `identity:create_grant(project_id=93d502514f39421c8d8ce22989785c76, user_id=09903e8bf45e4839b5fcd01b272e4813, role_id=39e5c666285a42fe8fc2c3afca6260d1)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:49:11.698573 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bde2bd4-b6be-4fc5-a494-a64f98fe3902 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:49:11.710521 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1424/2850] 10.222.0.33 () {60 vars in 1482 bytes} [Mon Jul 25 05:49:11 2022] PUT /identity/v3/projects/93d502514f39421c8d8ce22989785c76/users/09903e8bf45e4839b5fcd01b272e4813/roles/39e5c666285a42fe8fc2c3afca6260d1 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 25 05:49:11.721949 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69320a02-0329-400d-8da0-9d35513b2a67 None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:49:11.721949 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69320a02-0329-400d-8da0-9d35513b2a67 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:49:11.721949 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69320a02-0329-400d-8da0-9d35513b2a67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:49:11.748957 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-69320a02-0329-400d-8da0-9d35513b2a67 None None] MFA Rules not processed for user `09903e8bf45e4839b5fcd01b272e4813`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:49:11.750108 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-69320a02-0329-400d-8da0-9d35513b2a67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:49:11.822258 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1427/2851] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 25 05:49:11 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:49:11.833636 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bde2bd4-b6be-4fc5-a494-a64f98fe3902 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:49:11.835380 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-9bde2bd4-b6be-4fc5-a494-a64f98fe3902 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:49:11.892137 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bde2bd4-b6be-4fc5-a494-a64f98fe3902 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:49:11.896584 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8b007a-500c-469b-a3a0-fc373d228c03 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db57b478c25643dbaf936f90f6e17d0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92afba266cce4d1bad4134c351e74be3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:49:11.897267 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8b007a-500c-469b-a3a0-fc373d228c03 service cinder] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:49:11.897554 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8b007a-500c-469b-a3a0-fc373d228c03 service cinder] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:49:11.897831 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8b007a-500c-469b-a3a0-fc373d228c03 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:49:11.906653 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a8b007a-500c-469b-a3a0-fc373d228c03 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:49:11.908805 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a8b007a-500c-469b-a3a0-fc373d228c03 service cinder] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:49:11.918365 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1425/2852] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jul 25 05:49:11 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 05:49:12.266800 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69320a02-0329-400d-8da0-9d35513b2a67 None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:49:12.328842 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69320a02-0329-400d-8da0-9d35513b2a67 None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:49:12.333396 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a15044b9-93ec-4cce-8433-f88913724772 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:49:12.334255 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15044b9-93ec-4cce-8433-f88913724772 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:49:12.334653 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15044b9-93ec-4cce-8433-f88913724772 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:49:12.334935 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15044b9-93ec-4cce-8433-f88913724772 admin admin] PATH_INFO: `/v3/users/d59eec85f359439a920d84494a70906d` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:49:12.336114 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a15044b9-93ec-4cce-8433-f88913724772 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d59eec85f359439a920d84494a70906d)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:49:12.338055 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a15044b9-93ec-4cce-8433-f88913724772 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:49:12.382753 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-a15044b9-93ec-4cce-8433-f88913724772 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd59eec85f359439a920d84494a70906d'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:49:12.383919 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:49:12.396036 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-a15044b9-93ec-4cce-8433-f88913724772 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd59eec85f359439a920d84494a70906d'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:49:12.396699 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-a15044b9-93ec-4cce-8433-f88913724772 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd59eec85f359439a920d84494a70906d'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:49:12.403799 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-a15044b9-93ec-4cce-8433-f88913724772 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd59eec85f359439a920d84494a70906d'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:49:12.404413 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-a15044b9-93ec-4cce-8433-f88913724772 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd59eec85f359439a920d84494a70906d'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:49:12.416244 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1428/2853] 10.222.0.33 () {58 vars in 1223 bytes} [Mon Jul 25 05:49:12 2022] DELETE /identity/v3/users/d59eec85f359439a920d84494a70906d => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 05:49:12.420770 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b4aa37-677b-462f-a0ea-e291eba594e0 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:49:12.421094 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b4aa37-677b-462f-a0ea-e291eba594e0 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:49:12.421383 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b4aa37-677b-462f-a0ea-e291eba594e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:49:12.439543 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-70b4aa37-677b-462f-a0ea-e291eba594e0 None None] MFA Rules not processed for user `1eae540242d94c10b0ad33960a855f08`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:49:12.441423 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-70b4aa37-677b-462f-a0ea-e291eba594e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:49:12.499092 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1426/2854] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 25 05:49:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:49:12.509314 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a15044b9-93ec-4cce-8433-f88913724772 admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:49:12.510729 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-a15044b9-93ec-4cce-8433-f88913724772 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:49:12.566770 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a15044b9-93ec-4cce-8433-f88913724772 admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:49:12.570378 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c3439a9-7205-448b-8573-f0aac52da719 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b2931200f99e43d79a50a9f6e02738e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92afba266cce4d1bad4134c351e74be3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:49:12.571014 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3439a9-7205-448b-8573-f0aac52da719 service neutron] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:49:12.571295 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3439a9-7205-448b-8573-f0aac52da719 service neutron] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:49:12.571570 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3439a9-7205-448b-8573-f0aac52da719 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:49:12.580336 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c3439a9-7205-448b-8573-f0aac52da719 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:49:12.581912 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c3439a9-7205-448b-8573-f0aac52da719 service neutron] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:49:12.591535 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1429/2855] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jul 25 05:49:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 05:49:12.859275 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b4aa37-677b-462f-a0ea-e291eba594e0 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:49:12.860820 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-70b4aa37-677b-462f-a0ea-e291eba594e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:49:12.873614 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b4aa37-677b-462f-a0ea-e291eba594e0 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:49:12.879526 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c07757b2-a24f-404a-9c96-d2aa81407e13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:49:12.880397 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07757b2-a24f-404a-9c96-d2aa81407e13 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:49:12.880687 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07757b2-a24f-404a-9c96-d2aa81407e13 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:49:12.881004 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07757b2-a24f-404a-9c96-d2aa81407e13 admin admin] PATH_INFO: `/v3/projects/1755f8a133a6431494f0fe81d5a4cf7f` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:49:12.882211 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c07757b2-a24f-404a-9c96-d2aa81407e13 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1755f8a133a6431494f0fe81d5a4cf7f)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:49:12.884235 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c07757b2-a24f-404a-9c96-d2aa81407e13 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:49:12.939212 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-c07757b2-a24f-404a-9c96-d2aa81407e13 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1755f8a133a6431494f0fe81d5a4cf7f', 'request_id': 'req-c07757b2-a24f-404a-9c96-d2aa81407e13'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:49:12.945542 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-c07757b2-a24f-404a-9c96-d2aa81407e13 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1755f8a133a6431494f0fe81d5a4cf7f', 'request_id': 'req-c07757b2-a24f-404a-9c96-d2aa81407e13'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:49:12.951753 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-c07757b2-a24f-404a-9c96-d2aa81407e13 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1755f8a133a6431494f0fe81d5a4cf7f', 'request_id': 'req-c07757b2-a24f-404a-9c96-d2aa81407e13'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:49:12.965579 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-c07757b2-a24f-404a-9c96-d2aa81407e13 admin admin] The token cache is being invalidate because project 1755f8a133a6431494f0fe81d5a4cf7f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80249) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 05:49:12.965865 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-c07757b2-a24f-404a-9c96-d2aa81407e13 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:49:12.966478 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-c07757b2-a24f-404a-9c96-d2aa81407e13 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:49:12.968008 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1427/2856] 10.222.0.33 () {58 vars in 1232 bytes} [Mon Jul 25 05:49:12 2022] DELETE /identity/v3/projects/1755f8a133a6431494f0fe81d5a4cf7f => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 05:49:12.972636 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c3439a9-7205-448b-8573-f0aac52da719 service neutron] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:49:12.983087 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c3439a9-7205-448b-8573-f0aac52da719 service neutron] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:49:13.041063 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-857ac03f-c7ae-44c8-8972-1c6575d175d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:49:13.041938 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857ac03f-c7ae-44c8-8972-1c6575d175d9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:49:13.042225 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857ac03f-c7ae-44c8-8972-1c6575d175d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:49:13.042503 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857ac03f-c7ae-44c8-8972-1c6575d175d9 admin admin] PATH_INFO: `/v3/users/09903e8bf45e4839b5fcd01b272e4813` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:49:13.043683 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-857ac03f-c7ae-44c8-8972-1c6575d175d9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=09903e8bf45e4839b5fcd01b272e4813)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:49:13.045675 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-857ac03f-c7ae-44c8-8972-1c6575d175d9 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:49:13.098314 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-857ac03f-c7ae-44c8-8972-1c6575d175d9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '09903e8bf45e4839b5fcd01b272e4813'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:49:13.111503 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-857ac03f-c7ae-44c8-8972-1c6575d175d9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '09903e8bf45e4839b5fcd01b272e4813'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:49:13.112724 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-857ac03f-c7ae-44c8-8972-1c6575d175d9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '09903e8bf45e4839b5fcd01b272e4813'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:49:13.116635 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:49:13.121566 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-857ac03f-c7ae-44c8-8972-1c6575d175d9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '09903e8bf45e4839b5fcd01b272e4813'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:49:13.122414 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-857ac03f-c7ae-44c8-8972-1c6575d175d9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '09903e8bf45e4839b5fcd01b272e4813'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:49:13.138377 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1430/2857] 10.222.0.33 () {58 vars in 1223 bytes} [Mon Jul 25 05:49:12 2022] DELETE /identity/v3/users/09903e8bf45e4839b5fcd01b272e4813 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 25 05:49:13.447702 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c07757b2-a24f-404a-9c96-d2aa81407e13 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:49:13.450424 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-c07757b2-a24f-404a-9c96-d2aa81407e13 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:49:13.465760 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c07757b2-a24f-404a-9c96-d2aa81407e13 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:49:13.485750 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:49:13.547548 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5907ec85-c7f0-4b24-a022-6b0255337150 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:49:13.548925 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5907ec85-c7f0-4b24-a022-6b0255337150 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:49:13.549407 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5907ec85-c7f0-4b24-a022-6b0255337150 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:49:13.550471 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5907ec85-c7f0-4b24-a022-6b0255337150 admin admin] PATH_INFO: `/v3/projects/93d502514f39421c8d8ce22989785c76` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:49:13.552434 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5907ec85-c7f0-4b24-a022-6b0255337150 admin admin] RBAC: Authorizing `identity:delete_project(project_id=93d502514f39421c8d8ce22989785c76)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:49:13.555233 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5907ec85-c7f0-4b24-a022-6b0255337150 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:49:13.616424 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-5907ec85-c7f0-4b24-a022-6b0255337150 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '93d502514f39421c8d8ce22989785c76', 'request_id': 'req-5907ec85-c7f0-4b24-a022-6b0255337150'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:49:13.622872 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-5907ec85-c7f0-4b24-a022-6b0255337150 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '93d502514f39421c8d8ce22989785c76', 'request_id': 'req-5907ec85-c7f0-4b24-a022-6b0255337150'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:49:13.629767 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-5907ec85-c7f0-4b24-a022-6b0255337150 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '93d502514f39421c8d8ce22989785c76', 'request_id': 'req-5907ec85-c7f0-4b24-a022-6b0255337150'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:49:13.647219 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-5907ec85-c7f0-4b24-a022-6b0255337150 admin admin] The token cache is being invalidate because project 93d502514f39421c8d8ce22989785c76 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80249) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 05:49:13.647520 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-5907ec85-c7f0-4b24-a022-6b0255337150 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:49:13.648165 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-5907ec85-c7f0-4b24-a022-6b0255337150 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:49:13.650367 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1428/2858] 10.222.0.33 () {58 vars in 1232 bytes} [Mon Jul 25 05:49:13 2022] DELETE /identity/v3/projects/93d502514f39421c8d8ce22989785c76 => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 05:49:13.655999 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f3952e-f371-4375-a739-9b6f4ec9d069 None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:49:13.656295 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f3952e-f371-4375-a739-9b6f4ec9d069 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:49:13.656578 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f3952e-f371-4375-a739-9b6f4ec9d069 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:49:13.678464 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-63f3952e-f371-4375-a739-9b6f4ec9d069 None None] MFA Rules not processed for user `1eae540242d94c10b0ad33960a855f08`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:49:13.679703 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-63f3952e-f371-4375-a739-9b6f4ec9d069 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:49:13.743721 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1431/2859] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 25 05:49:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 05:49:13.751740 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12549cc5-cf33-4aa8-9193-ec675d25281d None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:49:13.752033 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12549cc5-cf33-4aa8-9193-ec675d25281d None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:49:13.752314 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12549cc5-cf33-4aa8-9193-ec675d25281d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:49:13.772170 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-12549cc5-cf33-4aa8-9193-ec675d25281d None None] MFA Rules not processed for user `1eae540242d94c10b0ad33960a855f08`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:49:13.773442 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-12549cc5-cf33-4aa8-9193-ec675d25281d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:49:13.784410 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1429/2860] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 25 05:49:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:49:13.788610 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f3952e-f371-4375-a739-9b6f4ec9d069 None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:49:13.801583 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f3952e-f371-4375-a739-9b6f4ec9d069 None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:49:13.806250 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de7cdb35-b87f-4c4e-b783-7b18bfd2b5eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:49:13.806913 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7cdb35-b87f-4c4e-b783-7b18bfd2b5eb admin admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:49:13.807195 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7cdb35-b87f-4c4e-b783-7b18bfd2b5eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:49:13.807470 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7cdb35-b87f-4c4e-b783-7b18bfd2b5eb admin admin] PATH_INFO: `/v3/domains` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:49:13.808207 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de7cdb35-b87f-4c4e-b783-7b18bfd2b5eb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:49:13.808512 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de7cdb35-b87f-4c4e-b783-7b18bfd2b5eb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:49:13.810259 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de7cdb35-b87f-4c4e-b783-7b18bfd2b5eb admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:49:13.822514 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1432/2861] 10.222.0.33 () {58 vars in 1152 bytes} [Mon Jul 25 05:49:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:49:13.826974 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12549cc5-cf33-4aa8-9193-ec675d25281d None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:49:13.836872 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12549cc5-cf33-4aa8-9193-ec675d25281d None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:49:13.841526 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2150ca7-a680-4962-9d86-1a169010cc01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:49:13.842173 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2150ca7-a680-4962-9d86-1a169010cc01 admin admin] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:49:13.842461 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2150ca7-a680-4962-9d86-1a169010cc01 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:49:13.842740 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2150ca7-a680-4962-9d86-1a169010cc01 admin admin] PATH_INFO: `/v3/projects` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:49:13.843447 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2150ca7-a680-4962-9d86-1a169010cc01 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:49:13.845364 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2150ca7-a680-4962-9d86-1a169010cc01 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:49:13.871105 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1430/2862] 10.222.0.33 () {60 vars in 1152 bytes} [Mon Jul 25 05:49:13 2022] POST /identity/v3/projects => generated 385 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 05:49:13.875240 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de7cdb35-b87f-4c4e-b783-7b18bfd2b5eb admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:49:13.886047 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de7cdb35-b87f-4c4e-b783-7b18bfd2b5eb admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:49:13.940734 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99aa4cd4-44f9-41de-bd50-1fbabf54be97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:49:13.941436 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99aa4cd4-44f9-41de-bd50-1fbabf54be97 admin admin] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:49:13.941729 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99aa4cd4-44f9-41de-bd50-1fbabf54be97 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:49:13.942010 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99aa4cd4-44f9-41de-bd50-1fbabf54be97 admin admin] PATH_INFO: `/v3/users` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:49:13.942705 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99aa4cd4-44f9-41de-bd50-1fbabf54be97 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:49:13.944622 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99aa4cd4-44f9-41de-bd50-1fbabf54be97 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:49:13.964864 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1433/2863] 10.222.0.33 () {60 vars in 1143 bytes} [Mon Jul 25 05:49:13 2022] POST /identity/v3/users => generated 293 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 05:49:13.968917 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2150ca7-a680-4962-9d86-1a169010cc01 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:49:13.980145 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2150ca7-a680-4962-9d86-1a169010cc01 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:49:13.984853 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-411acf65-bae3-4f5c-93f5-06c0daf7a7f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:49:13.985517 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411acf65-bae3-4f5c-93f5-06c0daf7a7f3 admin admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:49:13.985808 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411acf65-bae3-4f5c-93f5-06c0daf7a7f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:49:13.986107 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411acf65-bae3-4f5c-93f5-06c0daf7a7f3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:49:13.986853 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-411acf65-bae3-4f5c-93f5-06c0daf7a7f3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:49:13.988445 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-411acf65-bae3-4f5c-93f5-06c0daf7a7f3 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:49:13.998912 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1431/2864] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jul 25 05:49:13 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 05:49:14.002811 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99aa4cd4-44f9-41de-bd50-1fbabf54be97 admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:49:14.012615 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99aa4cd4-44f9-41de-bd50-1fbabf54be97 admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:49:14.017198 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1304eb23-d719-4357-b515-3eabec430789 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:49:14.017849 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1304eb23-d719-4357-b515-3eabec430789 admin admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:49:14.018667 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1304eb23-d719-4357-b515-3eabec430789 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:49:14.018945 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1304eb23-d719-4357-b515-3eabec430789 admin admin] PATH_INFO: `/v3/roles` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:49:14.019677 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1304eb23-d719-4357-b515-3eabec430789 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:49:14.021290 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1304eb23-d719-4357-b515-3eabec430789 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:49:14.031085 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1434/2865] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jul 25 05:49:14 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 05:49:14.035725 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-411acf65-bae3-4f5c-93f5-06c0daf7a7f3 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:49:14.045656 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-411acf65-bae3-4f5c-93f5-06c0daf7a7f3 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:49:14.050893 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ef892e-3668-4059-89ac-f7ad43162fb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:49:14.051818 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ef892e-3668-4059-89ac-f7ad43162fb7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:49:14.052042 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ef892e-3668-4059-89ac-f7ad43162fb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:49:14.052398 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ef892e-3668-4059-89ac-f7ad43162fb7 admin admin] PATH_INFO: `/v3/projects/b8e525ba77ee440c9040a845c98fefc7/users/ebf642c00241429c84dae401c2f91a23/roles/5074e74e35f54d4b9bb415c34999a82a` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:49:14.067322 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3ef892e-3668-4059-89ac-f7ad43162fb7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b8e525ba77ee440c9040a845c98fefc7, user_id=ebf642c00241429c84dae401c2f91a23, role_id=5074e74e35f54d4b9bb415c34999a82a)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:49:14.070072 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3ef892e-3668-4059-89ac-f7ad43162fb7 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:49:14.083370 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1432/2866] 10.222.0.33 () {60 vars in 1482 bytes} [Mon Jul 25 05:49:14 2022] PUT /identity/v3/projects/b8e525ba77ee440c9040a845c98fefc7/users/ebf642c00241429c84dae401c2f91a23/roles/5074e74e35f54d4b9bb415c34999a82a => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 05:49:14.145635 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d64473-b75a-4d35-a09d-43f9135686a6 None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:49:14.145937 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d64473-b75a-4d35-a09d-43f9135686a6 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:49:14.146226 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d64473-b75a-4d35-a09d-43f9135686a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:49:14.172311 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-61d64473-b75a-4d35-a09d-43f9135686a6 None None] MFA Rules not processed for user `ebf642c00241429c84dae401c2f91a23`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:49:14.173562 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-61d64473-b75a-4d35-a09d-43f9135686a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:49:14.217369 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:49:14.241382 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1435/2867] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 25 05:49:14 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:49:14.252296 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ef892e-3668-4059-89ac-f7ad43162fb7 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:49:14.254753 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-d3ef892e-3668-4059-89ac-f7ad43162fb7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:49:14.299478 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:49:14.312664 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ef892e-3668-4059-89ac-f7ad43162fb7 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:49:14.316828 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56822ffa-4cee-4c93-893f-73bd36f225c6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db57b478c25643dbaf936f90f6e17d0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92afba266cce4d1bad4134c351e74be3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:49:14.317474 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56822ffa-4cee-4c93-893f-73bd36f225c6 service cinder] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:49:14.317757 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56822ffa-4cee-4c93-893f-73bd36f225c6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:49:14.318032 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56822ffa-4cee-4c93-893f-73bd36f225c6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:49:14.326386 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56822ffa-4cee-4c93-893f-73bd36f225c6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:49:14.327805 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56822ffa-4cee-4c93-893f-73bd36f225c6 service cinder] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:49:14.337093 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1433/2868] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jul 25 05:49:14 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 05:49:21.660004 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d64473-b75a-4d35-a09d-43f9135686a6 None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:49:21.722484 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d64473-b75a-4d35-a09d-43f9135686a6 None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:49:21.727282 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6585a35a-16c9-4b12-a91b-0e1ee0d246c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:49:21.728149 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6585a35a-16c9-4b12-a91b-0e1ee0d246c7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:49:21.728439 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6585a35a-16c9-4b12-a91b-0e1ee0d246c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:49:21.728722 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6585a35a-16c9-4b12-a91b-0e1ee0d246c7 admin admin] PATH_INFO: `/v3/users/ebf642c00241429c84dae401c2f91a23` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:49:21.729948 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6585a35a-16c9-4b12-a91b-0e1ee0d246c7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ebf642c00241429c84dae401c2f91a23)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:49:21.731867 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6585a35a-16c9-4b12-a91b-0e1ee0d246c7 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:49:21.776325 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-6585a35a-16c9-4b12-a91b-0e1ee0d246c7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ebf642c00241429c84dae401c2f91a23'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:49:21.788595 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-6585a35a-16c9-4b12-a91b-0e1ee0d246c7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ebf642c00241429c84dae401c2f91a23'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:49:21.789286 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-6585a35a-16c9-4b12-a91b-0e1ee0d246c7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ebf642c00241429c84dae401c2f91a23'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:49:21.796205 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-6585a35a-16c9-4b12-a91b-0e1ee0d246c7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ebf642c00241429c84dae401c2f91a23'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:49:21.796915 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-6585a35a-16c9-4b12-a91b-0e1ee0d246c7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ebf642c00241429c84dae401c2f91a23'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:49:21.810273 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1436/2869] 10.222.0.33 () {58 vars in 1223 bytes} [Mon Jul 25 05:49:21 2022] DELETE /identity/v3/users/ebf642c00241429c84dae401c2f91a23 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 05:49:21.815425 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b064603-8e3e-49a3-9732-962b6342800f None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:49:21.815996 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b064603-8e3e-49a3-9732-962b6342800f None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:49:21.815996 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b064603-8e3e-49a3-9732-962b6342800f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:49:21.833677 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-3b064603-8e3e-49a3-9732-962b6342800f None None] MFA Rules not processed for user `1eae540242d94c10b0ad33960a855f08`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:49:21.834876 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-3b064603-8e3e-49a3-9732-962b6342800f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:49:21.894241 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1434/2870] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 25 05:49:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:49:21.903762 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6585a35a-16c9-4b12-a91b-0e1ee0d246c7 admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:49:21.905248 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-6585a35a-16c9-4b12-a91b-0e1ee0d246c7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:49:21.962099 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6585a35a-16c9-4b12-a91b-0e1ee0d246c7 admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:49:21.965762 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ad655a-6597-46c4-9a3b-eed9c8a591b9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b2931200f99e43d79a50a9f6e02738e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92afba266cce4d1bad4134c351e74be3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:49:21.966408 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ad655a-6597-46c4-9a3b-eed9c8a591b9 service neutron] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:49:21.966698 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ad655a-6597-46c4-9a3b-eed9c8a591b9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:49:21.966979 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ad655a-6597-46c4-9a3b-eed9c8a591b9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:49:21.976299 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60ad655a-6597-46c4-9a3b-eed9c8a591b9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:49:21.977765 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60ad655a-6597-46c4-9a3b-eed9c8a591b9 service neutron] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:49:21.987795 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1437/2871] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jul 25 05:49:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 05:49:22.258151 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b064603-8e3e-49a3-9732-962b6342800f None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:49:22.259755 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-3b064603-8e3e-49a3-9732-962b6342800f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:49:22.272232 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b064603-8e3e-49a3-9732-962b6342800f None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:49:22.277004 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ad9c73-252a-4ff9-b75d-933991c2381f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:49:22.277865 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ad9c73-252a-4ff9-b75d-933991c2381f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:49:22.278872 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ad9c73-252a-4ff9-b75d-933991c2381f admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:49:22.279202 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ad9c73-252a-4ff9-b75d-933991c2381f admin admin] PATH_INFO: `/v3/projects/b8e525ba77ee440c9040a845c98fefc7` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:49:22.280628 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35ad9c73-252a-4ff9-b75d-933991c2381f admin admin] RBAC: Authorizing `identity:delete_project(project_id=b8e525ba77ee440c9040a845c98fefc7)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:49:22.282645 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35ad9c73-252a-4ff9-b75d-933991c2381f admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:49:22.333413 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-35ad9c73-252a-4ff9-b75d-933991c2381f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b8e525ba77ee440c9040a845c98fefc7', 'request_id': 'req-35ad9c73-252a-4ff9-b75d-933991c2381f'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:49:22.338839 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-35ad9c73-252a-4ff9-b75d-933991c2381f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b8e525ba77ee440c9040a845c98fefc7', 'request_id': 'req-35ad9c73-252a-4ff9-b75d-933991c2381f'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:49:22.344791 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-35ad9c73-252a-4ff9-b75d-933991c2381f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b8e525ba77ee440c9040a845c98fefc7', 'request_id': 'req-35ad9c73-252a-4ff9-b75d-933991c2381f'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:49:22.357776 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-35ad9c73-252a-4ff9-b75d-933991c2381f admin admin] The token cache is being invalidate because project b8e525ba77ee440c9040a845c98fefc7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80249) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 05:49:22.358066 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-35ad9c73-252a-4ff9-b75d-933991c2381f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:49:22.358688 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-35ad9c73-252a-4ff9-b75d-933991c2381f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:49:22.360223 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1435/2872] 10.222.0.33 () {58 vars in 1232 bytes} [Mon Jul 25 05:49:22 2022] DELETE /identity/v3/projects/b8e525ba77ee440c9040a845c98fefc7 => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 05:49:22.365731 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491c40c3-8b3d-4446-acb3-f9f75d7e2f31 None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:49:22.366024 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491c40c3-8b3d-4446-acb3-f9f75d7e2f31 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:49:22.366306 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491c40c3-8b3d-4446-acb3-f9f75d7e2f31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:49:22.373606 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:49:22.386235 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-491c40c3-8b3d-4446-acb3-f9f75d7e2f31 None None] MFA Rules not processed for user `1eae540242d94c10b0ad33960a855f08`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:49:22.387459 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-491c40c3-8b3d-4446-acb3-f9f75d7e2f31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:49:22.445838 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1438/2873] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 25 05:49:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:49:22.453641 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49305049-6505-462a-806d-1a7f0b5d01fa None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:49:22.453937 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49305049-6505-462a-806d-1a7f0b5d01fa None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:49:22.454219 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49305049-6505-462a-806d-1a7f0b5d01fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:49:22.473959 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-49305049-6505-462a-806d-1a7f0b5d01fa None None] MFA Rules not processed for user `1eae540242d94c10b0ad33960a855f08`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:49:22.475170 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-49305049-6505-462a-806d-1a7f0b5d01fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:49:22.483121 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1436/2874] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 25 05:49:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:49:22.488317 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-491c40c3-8b3d-4446-acb3-f9f75d7e2f31 None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:49:22.498016 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-491c40c3-8b3d-4446-acb3-f9f75d7e2f31 None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:49:22.503012 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4340d1c4-526c-407e-9b22-dee062c91f07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:49:22.503664 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4340d1c4-526c-407e-9b22-dee062c91f07 admin admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:49:22.503952 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4340d1c4-526c-407e-9b22-dee062c91f07 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:49:22.504233 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4340d1c4-526c-407e-9b22-dee062c91f07 admin admin] PATH_INFO: `/v3/domains` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:49:22.505010 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4340d1c4-526c-407e-9b22-dee062c91f07 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:49:22.505324 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4340d1c4-526c-407e-9b22-dee062c91f07 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:49:22.506916 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4340d1c4-526c-407e-9b22-dee062c91f07 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:49:22.518752 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1439/2875] 10.222.0.33 () {58 vars in 1152 bytes} [Mon Jul 25 05:49:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:49:22.522997 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49305049-6505-462a-806d-1a7f0b5d01fa None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:49:22.532179 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49305049-6505-462a-806d-1a7f0b5d01fa None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:49:22.537491 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-046abe2b-45ad-41da-a73f-f25cf9575893 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:49:22.538139 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046abe2b-45ad-41da-a73f-f25cf9575893 admin admin] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:49:22.538429 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046abe2b-45ad-41da-a73f-f25cf9575893 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:49:22.538743 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046abe2b-45ad-41da-a73f-f25cf9575893 admin admin] PATH_INFO: `/v3/projects` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:49:22.539448 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-046abe2b-45ad-41da-a73f-f25cf9575893 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:49:22.541410 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-046abe2b-45ad-41da-a73f-f25cf9575893 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:49:22.566622 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1437/2876] 10.222.0.33 () {60 vars in 1152 bytes} [Mon Jul 25 05:49:22 2022] POST /identity/v3/projects => generated 373 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 05:49:22.570881 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4340d1c4-526c-407e-9b22-dee062c91f07 admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:49:22.580714 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4340d1c4-526c-407e-9b22-dee062c91f07 admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:49:22.636607 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8696bf3-bd36-41e5-a2ad-1dc608cc78b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:49:22.637294 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8696bf3-bd36-41e5-a2ad-1dc608cc78b5 admin admin] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:49:22.637581 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8696bf3-bd36-41e5-a2ad-1dc608cc78b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:49:22.637862 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8696bf3-bd36-41e5-a2ad-1dc608cc78b5 admin admin] PATH_INFO: `/v3/users` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:49:22.638552 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8696bf3-bd36-41e5-a2ad-1dc608cc78b5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:49:22.640497 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8696bf3-bd36-41e5-a2ad-1dc608cc78b5 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:49:22.661872 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1440/2877] 10.222.0.33 () {60 vars in 1143 bytes} [Mon Jul 25 05:49:22 2022] POST /identity/v3/users => generated 287 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 05:49:22.666085 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-046abe2b-45ad-41da-a73f-f25cf9575893 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:49:22.676968 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-046abe2b-45ad-41da-a73f-f25cf9575893 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:49:22.681855 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ae41fc-4130-45f0-96c0-fe561cdb4744 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:49:22.682524 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ae41fc-4130-45f0-96c0-fe561cdb4744 admin admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:49:22.682806 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ae41fc-4130-45f0-96c0-fe561cdb4744 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:49:22.683084 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ae41fc-4130-45f0-96c0-fe561cdb4744 admin admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:49:22.683815 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19ae41fc-4130-45f0-96c0-fe561cdb4744 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:49:22.685450 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19ae41fc-4130-45f0-96c0-fe561cdb4744 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:49:22.697148 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1438/2878] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jul 25 05:49:22 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 25 05:49:22.701297 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8696bf3-bd36-41e5-a2ad-1dc608cc78b5 admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:49:22.712194 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8696bf3-bd36-41e5-a2ad-1dc608cc78b5 admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:49:22.716741 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e3e15e4-432e-4edf-bbee-836e184991c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:49:22.717422 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3e15e4-432e-4edf-bbee-836e184991c0 admin admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:49:22.717708 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3e15e4-432e-4edf-bbee-836e184991c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:49:22.717985 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3e15e4-432e-4edf-bbee-836e184991c0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:49:22.718706 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e3e15e4-432e-4edf-bbee-836e184991c0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:49:22.720289 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e3e15e4-432e-4edf-bbee-836e184991c0 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:49:22.731564 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1441/2879] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jul 25 05:49:22 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 05:49:22.735739 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ae41fc-4130-45f0-96c0-fe561cdb4744 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:49:22.746507 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ae41fc-4130-45f0-96c0-fe561cdb4744 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:49:22.751433 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f974a9-308e-4d7d-8eba-751c638be008 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:49:22.752321 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f974a9-308e-4d7d-8eba-751c638be008 admin admin] REQUEST_METHOD: `PUT` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:49:22.752596 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f974a9-308e-4d7d-8eba-751c638be008 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:49:22.752901 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f974a9-308e-4d7d-8eba-751c638be008 admin admin] PATH_INFO: `/v3/projects/c8a1790f9a76478eab9dd2bbe8941b8f/users/010a94640af34bd18e5a873af1527bb0/roles/5074e74e35f54d4b9bb415c34999a82a` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:49:22.767236 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5f974a9-308e-4d7d-8eba-751c638be008 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c8a1790f9a76478eab9dd2bbe8941b8f, user_id=010a94640af34bd18e5a873af1527bb0, role_id=5074e74e35f54d4b9bb415c34999a82a)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:49:22.770555 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5f974a9-308e-4d7d-8eba-751c638be008 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:49:22.782969 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1439/2880] 10.222.0.33 () {60 vars in 1482 bytes} [Mon Jul 25 05:49:22 2022] PUT /identity/v3/projects/c8a1790f9a76478eab9dd2bbe8941b8f/users/010a94640af34bd18e5a873af1527bb0/roles/5074e74e35f54d4b9bb415c34999a82a => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 05:49:22.788662 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2241bf-ff90-4c89-bd94-b34ce555c509 None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:49:22.789209 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2241bf-ff90-4c89-bd94-b34ce555c509 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:49:22.789556 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2241bf-ff90-4c89-bd94-b34ce555c509 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:49:22.807301 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-ca2241bf-ff90-4c89-bd94-b34ce555c509 None None] MFA Rules not processed for user `1eae540242d94c10b0ad33960a855f08`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:49:22.808835 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-ca2241bf-ff90-4c89-bd94-b34ce555c509 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:49:22.868188 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1442/2881] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 25 05:49:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:49:22.878647 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f974a9-308e-4d7d-8eba-751c638be008 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:49:22.880440 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-f5f974a9-308e-4d7d-8eba-751c638be008 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:49:22.886960 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:49:22.937262 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f974a9-308e-4d7d-8eba-751c638be008 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:49:22.941556 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-071dba37-5365-4226-956b-4f5278fd40e0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b2931200f99e43d79a50a9f6e02738e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92afba266cce4d1bad4134c351e74be3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:49:22.942194 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071dba37-5365-4226-956b-4f5278fd40e0 service neutron] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:49:22.942478 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071dba37-5365-4226-956b-4f5278fd40e0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:49:22.942756 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071dba37-5365-4226-956b-4f5278fd40e0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:49:22.951836 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-071dba37-5365-4226-956b-4f5278fd40e0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:49:22.953259 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-071dba37-5365-4226-956b-4f5278fd40e0 service neutron] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:49:22.962510 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1440/2882] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jul 25 05:49:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 05:49:23.573437 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f387cb4-1b0f-4d88-abb0-cfbe7abc04df None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:49:23.573733 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f387cb4-1b0f-4d88-abb0-cfbe7abc04df None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:49:23.574020 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f387cb4-1b0f-4d88-abb0-cfbe7abc04df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:49:23.717864 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:49:23.727216 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-7f387cb4-1b0f-4d88-abb0-cfbe7abc04df None None] MFA Rules not processed for user `010a94640af34bd18e5a873af1527bb0`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:49:23.728608 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-7f387cb4-1b0f-4d88-abb0-cfbe7abc04df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:49:23.796539 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1443/2883] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 25 05:49:23 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 224 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:49:23.808432 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-071dba37-5365-4226-956b-4f5278fd40e0 service neutron] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:49:23.809904 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-071dba37-5365-4226-956b-4f5278fd40e0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:49:23.867887 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-071dba37-5365-4226-956b-4f5278fd40e0 service neutron] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:49:23.871488 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be6c0bd7-9091-4cf1-a391-8d59d4abf0cb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db57b478c25643dbaf936f90f6e17d0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92afba266cce4d1bad4134c351e74be3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:49:23.872132 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6c0bd7-9091-4cf1-a391-8d59d4abf0cb service cinder] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:49:23.872421 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6c0bd7-9091-4cf1-a391-8d59d4abf0cb service cinder] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:49:23.872700 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6c0bd7-9091-4cf1-a391-8d59d4abf0cb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:49:23.881126 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be6c0bd7-9091-4cf1-a391-8d59d4abf0cb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:49:23.882559 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be6c0bd7-9091-4cf1-a391-8d59d4abf0cb service cinder] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:49:23.891409 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1441/2884] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jul 25 05:49:23 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 05:49:25.248508 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c8bb66-1fb8-4aaa-9ffe-70d51bcc96c0 None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:49:25.249026 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c8bb66-1fb8-4aaa-9ffe-70d51bcc96c0 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:49:25.249146 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c8bb66-1fb8-4aaa-9ffe-70d51bcc96c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:49:25.266710 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-01c8bb66-1fb8-4aaa-9ffe-70d51bcc96c0 None None] MFA Rules not processed for user `010a94640af34bd18e5a873af1527bb0`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:49:25.267932 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-01c8bb66-1fb8-4aaa-9ffe-70d51bcc96c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:49:25.274726 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1444/2885] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 25 05:49:25 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 27 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:49:25.506823 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be6c0bd7-9091-4cf1-a391-8d59d4abf0cb service cinder] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:49:25.508303 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-be6c0bd7-9091-4cf1-a391-8d59d4abf0cb service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:49:25.520904 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be6c0bd7-9091-4cf1-a391-8d59d4abf0cb service cinder] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:49:25.524346 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f55a54e-9591-4663-96fc-a9562c5edab4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b2931200f99e43d79a50a9f6e02738e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92afba266cce4d1bad4134c351e74be3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:49:25.525006 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f55a54e-9591-4663-96fc-a9562c5edab4 service neutron] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:49:25.525290 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f55a54e-9591-4663-96fc-a9562c5edab4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:49:25.525570 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f55a54e-9591-4663-96fc-a9562c5edab4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:49:25.527279 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-4f55a54e-9591-4663-96fc-a9562c5edab4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:49:25.536568 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f55a54e-9591-4663-96fc-a9562c5edab4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:49:25.538007 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f55a54e-9591-4663-96fc-a9562c5edab4 service neutron] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:49:25.545846 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1442/2886] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jul 25 05:49:25 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 05:49:35.261584 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=30 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:49:35.542229 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=56 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:49:39.269693 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0acd2625-e434-473d-9efc-20978622fd73 None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:49:39.270158 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0acd2625-e434-473d-9efc-20978622fd73 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:49:39.270269 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0acd2625-e434-473d-9efc-20978622fd73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:49:39.287512 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-0acd2625-e434-473d-9efc-20978622fd73 None None] MFA Rules not processed for user `010a94640af34bd18e5a873af1527bb0`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:49:39.288713 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-0acd2625-e434-473d-9efc-20978622fd73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:49:39.295508 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1445/2887] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 25 05:49:39 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 27 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:49:49.282877 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=3 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:49:58.066235 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f260326-78a1-4b6c-80b5-7b27de137f38 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:49:58.066765 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f260326-78a1-4b6c-80b5-7b27de137f38 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:49:58.066765 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f260326-78a1-4b6c-80b5-7b27de137f38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:49:58.084124 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-9f260326-78a1-4b6c-80b5-7b27de137f38 None None] MFA Rules not processed for user `3352531be1b54f76b864383ccc2e5ec2`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:49:58.085367 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-9f260326-78a1-4b6c-80b5-7b27de137f38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:49:58.140603 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1443/2888] 10.222.0.33 () {58 vars in 1009 bytes} [Mon Jul 25 05:49:58 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 05:49:58.185549 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0acd2625-e434-473d-9efc-20978622fd73 None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:49:58.187445 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-0acd2625-e434-473d-9efc-20978622fd73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:49:58.270233 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0acd2625-e434-473d-9efc-20978622fd73 None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:49:58.275339 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f53293e8-c701-44a1-b5f4-72009e91fe12 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb135a8d5d7444cfbfa336115ba2ba44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92afba266cce4d1bad4134c351e74be3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:49:58.275986 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53293e8-c701-44a1-b5f4-72009e91fe12 service swift] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:49:58.276270 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53293e8-c701-44a1-b5f4-72009e91fe12 service swift] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:49:58.276546 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53293e8-c701-44a1-b5f4-72009e91fe12 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:49:58.285401 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f53293e8-c701-44a1-b5f4-72009e91fe12 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:49:58.286957 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f53293e8-c701-44a1-b5f4-72009e91fe12 service swift] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:49:58.294694 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1446/2889] 10.222.0.33 () {60 vars in 1384 bytes} [Mon Jul 25 05:49:58 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 25 05:50:08.133796 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=25 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:50:08.291716 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=37 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:50:11.633401 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f260326-78a1-4b6c-80b5-7b27de137f38 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:50:11.635225 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-9f260326-78a1-4b6c-80b5-7b27de137f38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:50:11.743044 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f260326-78a1-4b6c-80b5-7b27de137f38 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:50:11.757988 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a719bc-2db7-4a9a-942e-0397c079ec3a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cd482a61e8a420db45bb532e5dae6e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92afba266cce4d1bad4134c351e74be3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:50:11.758678 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a719bc-2db7-4a9a-942e-0397c079ec3a service nova] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:50:11.759598 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a719bc-2db7-4a9a-942e-0397c079ec3a service nova] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:50:11.759885 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a719bc-2db7-4a9a-942e-0397c079ec3a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:50:11.763382 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-a3a719bc-2db7-4a9a-942e-0397c079ec3a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:50:11.787621 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3a719bc-2db7-4a9a-942e-0397c079ec3a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:50:11.789296 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3a719bc-2db7-4a9a-942e-0397c079ec3a service nova] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:50:11.818072 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1444/2890] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jul 25 05:50:11 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 184 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 05:50:12.710626 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d73fb76-a79e-4529-b3f7-fc67a92385d9 None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:50:12.711122 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d73fb76-a79e-4529-b3f7-fc67a92385d9 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:50:12.711288 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d73fb76-a79e-4529-b3f7-fc67a92385d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:50:12.730853 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-0d73fb76-a79e-4529-b3f7-fc67a92385d9 None None] MFA Rules not processed for user `3352531be1b54f76b864383ccc2e5ec2`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:50:12.732712 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-0d73fb76-a79e-4529-b3f7-fc67a92385d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:50:12.739893 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1447/2891] 10.222.0.33 () {58 vars in 967 bytes} [Mon Jul 25 05:50:12 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 05:50:12.751935 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a719bc-2db7-4a9a-942e-0397c079ec3a service nova] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:50:12.763385 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a719bc-2db7-4a9a-942e-0397c079ec3a service nova] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:50:12.768318 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74924de1-5804-4db1-b1cc-e04a646cb02f service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb135a8d5d7444cfbfa336115ba2ba44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92afba266cce4d1bad4134c351e74be3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:50:12.769110 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74924de1-5804-4db1-b1cc-e04a646cb02f service swift] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:50:12.769395 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74924de1-5804-4db1-b1cc-e04a646cb02f service swift] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:50:12.769671 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74924de1-5804-4db1-b1cc-e04a646cb02f service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:50:12.777858 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74924de1-5804-4db1-b1cc-e04a646cb02f service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:50:12.780003 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74924de1-5804-4db1-b1cc-e04a646cb02f service swift] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:50:12.788245 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1445/2892] 10.222.0.33 () {60 vars in 1384 bytes} [Mon Jul 25 05:50:12 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 25 05:50:14.827964 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d73fb76-a79e-4529-b3f7-fc67a92385d9 None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:50:14.830766 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-0d73fb76-a79e-4529-b3f7-fc67a92385d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:50:14.891247 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74924de1-5804-4db1-b1cc-e04a646cb02f service swift] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:50:14.902622 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74924de1-5804-4db1-b1cc-e04a646cb02f service swift] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:50:14.908205 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c651524f-db2f-4045-b21c-d3d9ef3c00d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:50:14.909293 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c651524f-db2f-4045-b21c-d3d9ef3c00d2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:50:14.909599 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c651524f-db2f-4045-b21c-d3d9ef3c00d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:50:14.909853 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c651524f-db2f-4045-b21c-d3d9ef3c00d2 admin admin] PATH_INFO: `/v3/users/010a94640af34bd18e5a873af1527bb0` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:50:14.911095 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c651524f-db2f-4045-b21c-d3d9ef3c00d2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=010a94640af34bd18e5a873af1527bb0)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:50:14.913052 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c651524f-db2f-4045-b21c-d3d9ef3c00d2 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:50:14.924188 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d73fb76-a79e-4529-b3f7-fc67a92385d9 None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:50:14.929531 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed7370c3-5658-4d9b-ab3b-0e8446742d0e service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '317790016e7047fe96af73925de5d872', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92afba266cce4d1bad4134c351e74be3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:50:14.931053 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7370c3-5658-4d9b-ab3b-0e8446742d0e service placement] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:50:14.931344 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7370c3-5658-4d9b-ab3b-0e8446742d0e service placement] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:50:14.931623 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7370c3-5658-4d9b-ab3b-0e8446742d0e service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:50:14.933467 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-ed7370c3-5658-4d9b-ab3b-0e8446742d0e service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:50:14.953990 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed7370c3-5658-4d9b-ab3b-0e8446742d0e service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:50:14.956623 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed7370c3-5658-4d9b-ab3b-0e8446742d0e service placement] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:50:14.968492 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1448/2893] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jul 25 05:50:14 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 05:50:14.976930 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-c651524f-db2f-4045-b21c-d3d9ef3c00d2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '010a94640af34bd18e5a873af1527bb0'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:50:14.977879 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-c651524f-db2f-4045-b21c-d3d9ef3c00d2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '010a94640af34bd18e5a873af1527bb0'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:50:14.986821 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-c651524f-db2f-4045-b21c-d3d9ef3c00d2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '010a94640af34bd18e5a873af1527bb0'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:50:15.000884 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-c651524f-db2f-4045-b21c-d3d9ef3c00d2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '010a94640af34bd18e5a873af1527bb0'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:50:15.002056 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-c651524f-db2f-4045-b21c-d3d9ef3c00d2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '010a94640af34bd18e5a873af1527bb0'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:50:15.017267 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1446/2894] 10.222.0.33 () {58 vars in 1223 bytes} [Mon Jul 25 05:50:14 2022] DELETE /identity/v3/users/010a94640af34bd18e5a873af1527bb0 => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 05:50:15.225641 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed7370c3-5658-4d9b-ab3b-0e8446742d0e service placement] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:50:15.236991 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed7370c3-5658-4d9b-ab3b-0e8446742d0e service placement] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:50:15.292528 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf19252-82ca-4ad9-98fb-c16941035667 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:50:15.293403 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf19252-82ca-4ad9-98fb-c16941035667 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:50:15.294373 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf19252-82ca-4ad9-98fb-c16941035667 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:50:15.294751 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf19252-82ca-4ad9-98fb-c16941035667 admin admin] PATH_INFO: `/v3/projects/c8a1790f9a76478eab9dd2bbe8941b8f` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:50:15.295930 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bf19252-82ca-4ad9-98fb-c16941035667 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c8a1790f9a76478eab9dd2bbe8941b8f)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:50:15.297861 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bf19252-82ca-4ad9-98fb-c16941035667 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:50:15.346417 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-3bf19252-82ca-4ad9-98fb-c16941035667 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c8a1790f9a76478eab9dd2bbe8941b8f', 'request_id': 'req-3bf19252-82ca-4ad9-98fb-c16941035667'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:50:15.351904 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-3bf19252-82ca-4ad9-98fb-c16941035667 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c8a1790f9a76478eab9dd2bbe8941b8f', 'request_id': 'req-3bf19252-82ca-4ad9-98fb-c16941035667'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:50:15.358712 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-3bf19252-82ca-4ad9-98fb-c16941035667 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c8a1790f9a76478eab9dd2bbe8941b8f', 'request_id': 'req-3bf19252-82ca-4ad9-98fb-c16941035667'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:50:15.370623 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-3bf19252-82ca-4ad9-98fb-c16941035667 admin admin] The token cache is being invalidate because project c8a1790f9a76478eab9dd2bbe8941b8f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80248) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 05:50:15.370912 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-3bf19252-82ca-4ad9-98fb-c16941035667 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:50:15.371550 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-3bf19252-82ca-4ad9-98fb-c16941035667 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:50:15.373381 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1449/2895] 10.222.0.33 () {58 vars in 1232 bytes} [Mon Jul 25 05:50:15 2022] DELETE /identity/v3/projects/c8a1790f9a76478eab9dd2bbe8941b8f => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 05:50:23.111143 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c651524f-db2f-4045-b21c-d3d9ef3c00d2 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:50:23.113482 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-c651524f-db2f-4045-b21c-d3d9ef3c00d2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:50:23.186740 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:50:23.199961 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c651524f-db2f-4045-b21c-d3d9ef3c00d2 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:50:23.255383 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-742c3080-37c0-436e-b018-e04b774c0624 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b2931200f99e43d79a50a9f6e02738e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92afba266cce4d1bad4134c351e74be3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:50:23.256036 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742c3080-37c0-436e-b018-e04b774c0624 service neutron] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:50:23.256317 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742c3080-37c0-436e-b018-e04b774c0624 service neutron] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:50:23.256591 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742c3080-37c0-436e-b018-e04b774c0624 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:50:23.258481 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-742c3080-37c0-436e-b018-e04b774c0624 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:50:23.319716 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-742c3080-37c0-436e-b018-e04b774c0624 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:50:23.321217 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-742c3080-37c0-436e-b018-e04b774c0624 service neutron] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:50:23.332602 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1447/2896] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jul 25 05:50:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 222 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 05:50:24.134880 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf19252-82ca-4ad9-98fb-c16941035667 admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:50:24.137536 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-3bf19252-82ca-4ad9-98fb-c16941035667 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:50:24.150160 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf19252-82ca-4ad9-98fb-c16941035667 admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:50:24.155054 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2166cd95-1d9f-4e53-b48d-24ef19a27944 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:50:24.155928 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2166cd95-1d9f-4e53-b48d-24ef19a27944 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:50:24.156215 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2166cd95-1d9f-4e53-b48d-24ef19a27944 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:50:24.156494 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2166cd95-1d9f-4e53-b48d-24ef19a27944 admin admin] PATH_INFO: `/v3/users/1574ee677bad49d190a0f82827d5aade` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:50:24.157737 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2166cd95-1d9f-4e53-b48d-24ef19a27944 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1574ee677bad49d190a0f82827d5aade)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:50:24.159642 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2166cd95-1d9f-4e53-b48d-24ef19a27944 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:50:24.165203 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:50:24.208958 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-2166cd95-1d9f-4e53-b48d-24ef19a27944 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1574ee677bad49d190a0f82827d5aade'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:50:24.221312 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-2166cd95-1d9f-4e53-b48d-24ef19a27944 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1574ee677bad49d190a0f82827d5aade'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:50:24.221945 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-2166cd95-1d9f-4e53-b48d-24ef19a27944 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1574ee677bad49d190a0f82827d5aade'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:50:24.228752 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-2166cd95-1d9f-4e53-b48d-24ef19a27944 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1574ee677bad49d190a0f82827d5aade'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:50:24.229424 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-2166cd95-1d9f-4e53-b48d-24ef19a27944 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1574ee677bad49d190a0f82827d5aade'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:50:24.242083 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1450/2897] 10.222.0.33 () {58 vars in 1223 bytes} [Mon Jul 25 05:50:24 2022] DELETE /identity/v3/users/1574ee677bad49d190a0f82827d5aade => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 05:50:24.417811 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-742c3080-37c0-436e-b018-e04b774c0624 service neutron] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:50:24.428372 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-742c3080-37c0-436e-b018-e04b774c0624 service neutron] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:50:24.490523 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-380e5554-d60f-4a7a-ad6e-0695968b978a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:50:24.491299 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380e5554-d60f-4a7a-ad6e-0695968b978a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:50:24.491581 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380e5554-d60f-4a7a-ad6e-0695968b978a admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:50:24.491857 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380e5554-d60f-4a7a-ad6e-0695968b978a admin admin] PATH_INFO: `/v3/projects/354d7ebd6b454fb5a601a2a78a88561e` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:50:24.493062 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-380e5554-d60f-4a7a-ad6e-0695968b978a admin admin] RBAC: Authorizing `identity:delete_project(project_id=354d7ebd6b454fb5a601a2a78a88561e)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:50:24.494959 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-380e5554-d60f-4a7a-ad6e-0695968b978a admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:50:24.540428 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-380e5554-d60f-4a7a-ad6e-0695968b978a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '354d7ebd6b454fb5a601a2a78a88561e', 'request_id': 'req-380e5554-d60f-4a7a-ad6e-0695968b978a'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:50:24.542958 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:50:24.547369 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-380e5554-d60f-4a7a-ad6e-0695968b978a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '354d7ebd6b454fb5a601a2a78a88561e', 'request_id': 'req-380e5554-d60f-4a7a-ad6e-0695968b978a'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:50:24.553702 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-380e5554-d60f-4a7a-ad6e-0695968b978a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '354d7ebd6b454fb5a601a2a78a88561e', 'request_id': 'req-380e5554-d60f-4a7a-ad6e-0695968b978a'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:50:24.565468 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-380e5554-d60f-4a7a-ad6e-0695968b978a admin admin] The token cache is being invalidate because project 354d7ebd6b454fb5a601a2a78a88561e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80249) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 05:50:24.565819 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-380e5554-d60f-4a7a-ad6e-0695968b978a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:50:24.566426 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-380e5554-d60f-4a7a-ad6e-0695968b978a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:50:24.567947 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1448/2898] 10.222.0.33 () {58 vars in 1232 bytes} [Mon Jul 25 05:50:24 2022] DELETE /identity/v3/projects/354d7ebd6b454fb5a601a2a78a88561e => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 25 05:50:24.572019 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2166cd95-1d9f-4e53-b48d-24ef19a27944 admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:50:24.573517 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-2166cd95-1d9f-4e53-b48d-24ef19a27944 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:50:24.586164 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2166cd95-1d9f-4e53-b48d-24ef19a27944 admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:50:24.642216 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15296ec4-1e16-4456-a04d-23e529b5df39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:50:24.643081 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15296ec4-1e16-4456-a04d-23e529b5df39 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:50:24.643373 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15296ec4-1e16-4456-a04d-23e529b5df39 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:50:24.643652 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15296ec4-1e16-4456-a04d-23e529b5df39 admin admin] PATH_INFO: `/v3/users/62cb0487409e4f36b227f5315e914d40` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:50:24.644883 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15296ec4-1e16-4456-a04d-23e529b5df39 admin admin] RBAC: Authorizing `identity:delete_user(user_id=62cb0487409e4f36b227f5315e914d40)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:50:24.646787 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15296ec4-1e16-4456-a04d-23e529b5df39 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:50:24.693172 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-15296ec4-1e16-4456-a04d-23e529b5df39 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '62cb0487409e4f36b227f5315e914d40'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:50:24.706791 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-15296ec4-1e16-4456-a04d-23e529b5df39 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '62cb0487409e4f36b227f5315e914d40'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:50:24.707495 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-15296ec4-1e16-4456-a04d-23e529b5df39 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '62cb0487409e4f36b227f5315e914d40'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:50:24.715091 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-15296ec4-1e16-4456-a04d-23e529b5df39 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '62cb0487409e4f36b227f5315e914d40'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:50:24.716119 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-15296ec4-1e16-4456-a04d-23e529b5df39 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '62cb0487409e4f36b227f5315e914d40'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:50:24.728204 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1451/2899] 10.222.0.33 () {58 vars in 1223 bytes} [Mon Jul 25 05:50:24 2022] DELETE /identity/v3/users/62cb0487409e4f36b227f5315e914d40 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 05:50:24.910113 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-380e5554-d60f-4a7a-ad6e-0695968b978a admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:50:24.911579 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-380e5554-d60f-4a7a-ad6e-0695968b978a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:50:24.925953 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-380e5554-d60f-4a7a-ad6e-0695968b978a admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:50:24.987908 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-464d1186-8720-465d-bc75-f179f2d999f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eae540242d94c10b0ad33960a855f08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85f4644d3bc94e89a0061a116652482e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:50:24.988792 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464d1186-8720-465d-bc75-f179f2d999f1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:50:24.989089 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464d1186-8720-465d-bc75-f179f2d999f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:50:24.989370 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464d1186-8720-465d-bc75-f179f2d999f1 admin admin] PATH_INFO: `/v3/projects/f647fa5b3b6f41b9a2bf80e202a2c629` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:50:24.990594 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-464d1186-8720-465d-bc75-f179f2d999f1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f647fa5b3b6f41b9a2bf80e202a2c629)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:50:24.992501 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-464d1186-8720-465d-bc75-f179f2d999f1 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:50:25.039924 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-464d1186-8720-465d-bc75-f179f2d999f1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f647fa5b3b6f41b9a2bf80e202a2c629', 'request_id': 'req-464d1186-8720-465d-bc75-f179f2d999f1'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:50:25.045322 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-464d1186-8720-465d-bc75-f179f2d999f1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f647fa5b3b6f41b9a2bf80e202a2c629', 'request_id': 'req-464d1186-8720-465d-bc75-f179f2d999f1'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:50:25.051328 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-464d1186-8720-465d-bc75-f179f2d999f1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f647fa5b3b6f41b9a2bf80e202a2c629', 'request_id': 'req-464d1186-8720-465d-bc75-f179f2d999f1'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:50:25.063738 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-464d1186-8720-465d-bc75-f179f2d999f1 admin admin] The token cache is being invalidate because project f647fa5b3b6f41b9a2bf80e202a2c629 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80249) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 05:50:25.064054 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-464d1186-8720-465d-bc75-f179f2d999f1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:50:25.064684 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-464d1186-8720-465d-bc75-f179f2d999f1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:50:25.066256 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1449/2900] 10.222.0.33 () {58 vars in 1232 bytes} [Mon Jul 25 05:50:24 2022] DELETE /identity/v3/projects/f647fa5b3b6f41b9a2bf80e202a2c629 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 05:50:34.724878 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=76,keystone:DELETE=20,keystone:INSERT=2 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:50:35.060618 dv-cinder-850830-1-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=57,keystone:DELETE=9,keystone:INSERT=2 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:51:22.992996 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15296ec4-1e16-4456-a04d-23e529b5df39 admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:51:22.994413 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-15296ec4-1e16-4456-a04d-23e529b5df39 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:51:23.062040 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15296ec4-1e16-4456-a04d-23e529b5df39 admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:51:23.119235 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e4eff4-c8fb-41c9-bcc6-7fe3af6d3cda service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '317790016e7047fe96af73925de5d872', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92afba266cce4d1bad4134c351e74be3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:51:23.119878 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e4eff4-c8fb-41c9-bcc6-7fe3af6d3cda service placement] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:51:23.120160 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e4eff4-c8fb-41c9-bcc6-7fe3af6d3cda service placement] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:51:23.120436 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e4eff4-c8fb-41c9-bcc6-7fe3af6d3cda service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:51:23.122234 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-d9e4eff4-c8fb-41c9-bcc6-7fe3af6d3cda service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:51:23.133145 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9e4eff4-c8fb-41c9-bcc6-7fe3af6d3cda service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:51:23.134695 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9e4eff4-c8fb-41c9-bcc6-7fe3af6d3cda service placement] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:51:23.145029 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1452/2901] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jul 25 05:51:22 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 25 05:51:33.140330 dv-cinder-850830-1-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=58 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}